[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:20 dist-test-slave-7kzw krb5kdc[29013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:20 dist-test-slave-7kzw krb5kdc[29013](info): set up 2 sockets
Aug 07 22:35:20 dist-test-slave-7kzw krb5kdc[29013](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 22:35:21.864790 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:21 dist-test-slave-7kzw krb5kdc[29013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:21.987471 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:44105
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:44105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:22.071736 29032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:22.071979 29032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:22.072021 29032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:22.073144 29032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:22.073210 29032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:22.073235 29032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:22.073266 29032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:22.073295 29032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:22.074837 29032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:44105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:44105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:22.075156 29032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:22.075428 29032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:22.082438 29043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:22.082923 29039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:22.083137 29041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:22.083564 29032 server_base.cc:1034] running on GCE node
I20260807 22:35:22.083829 29032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:22.084124 29032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:22.085325 29032 hybrid_clock.cc:648] HybridClock initialized: now 1786142122085327 us; error 86 us; skew 500 ppm
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:22.088006 29032 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:22.088482 29032 webserver.cc:460] Webserver started at http://127.28.82.254:36469/ using document root <none> and password file <none>
I20260807 22:35:22.088743 29032 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:22.088820 29032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:22.088976 29032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:22.090250 29032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "1cbf82b691f4441a999e93a7f80de6b9"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "1c1830265288821c01ac08390ee50428"
server_key_iv: "fdf50dff6b5c047e24d87257c2508d04"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.090960 29032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "1cbf82b691f4441a999e93a7f80de6b9"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "1c1830265288821c01ac08390ee50428"
server_key_iv: "fdf50dff6b5c047e24d87257c2508d04"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.092337 29032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:22.093101 29052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.093281 29032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:22.093390 29032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "1cbf82b691f4441a999e93a7f80de6b9"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "1c1830265288821c01ac08390ee50428"
server_key_iv: "fdf50dff6b5c047e24d87257c2508d04"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.093464 29032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:22.113185 29032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:22.115038 29032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:22.115192 29032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:22.119582 29032 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:44105
I20260807 22:35:22.119622 29132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:44105 every 8 connection(s)
I20260807 22:35:22.119976 29032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:22.120826 29133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:22.123550 29133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9: Bootstrap starting.
I20260807 22:35:22.123528 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29032
I20260807 22:35:22.123639 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:22.124344 29133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:22.124467 29003 external_mini_cluster.cc:1428] Setting key 36321a0c78a2a8362b86221324cf2e02
I20260807 22:35:22.124692 29133 log.cc:824] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:22.125501 29133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9: No bootstrap required, opened a new log
I20260807 22:35:22.129360 29133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbf82b691f4441a999e93a7f80de6b9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44105 } }
I20260807 22:35:22.129693 29133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:22.129740 29133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cbf82b691f4441a999e93a7f80de6b9, State: Initialized, Role: FOLLOWER
I20260807 22:35:22.129865 29133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbf82b691f4441a999e93a7f80de6b9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44105 } }
I20260807 22:35:22.129966 29133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:22.130000 29133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:22.130066 29133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:22.131615 29133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbf82b691f4441a999e93a7f80de6b9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44105 } }
I20260807 22:35:22.131796 29133 leader_election.cc:302] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cbf82b691f4441a999e93a7f80de6b9; no voters:
I20260807 22:35:22.131970 29133 leader_election.cc:288] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:22.132033 29143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:22.132208 29143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1cbf82b691f4441a999e93a7f80de6b9, State: Running, Role: LEADER
I20260807 22:35:22.132251 29133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:22.132310 29143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbf82b691f4441a999e93a7f80de6b9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44105 } }
I20260807 22:35:22.132835 29144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cbf82b691f4441a999e93a7f80de6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbf82b691f4441a999e93a7f80de6b9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44105 } } }
I20260807 22:35:22.132941 29144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:22.132835 29145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cbf82b691f4441a999e93a7f80de6b9. Latest consensus state: current_term: 1 leader_uuid: "1cbf82b691f4441a999e93a7f80de6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbf82b691f4441a999e93a7f80de6b9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44105 } } }
I20260807 22:35:22.133350 29145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:22.134656 29151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:22.135777 29151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:22.138262 29151 catalog_manager.cc:1347] Generated new cluster ID: 7c4246ac81e64cc993a103143220babb
I20260807 22:35:22.138358 29151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:22.147327 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.125438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38606 (local address 127.28.82.254:44105)
0807 22:35:22.125699 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:22.125708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:22.126049 (+ 341us) server_negotiation.cc:407] Connection header received
0807 22:35:22.130138 (+ 4089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:22.130145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:22.130199 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:22.130307 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:22.133561 (+ 3254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.136609 (+ 3048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.138569 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.139139 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.139314 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.143149 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:22.143153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:22.143153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:22.143173 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:22.145069 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.145500 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.145502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.145503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.145614 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.145895 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.145895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.145896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.146123 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:22.146403 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:22.147005 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 22:35:22.147127 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
I20260807 22:35:22.159257 29151 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:22.159818 29151 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:22.165925 29151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cbf82b691f4441a999e93a7f80de6b9: Generated new TSK 0
I20260807 22:35:22.166095 29151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:22.201429 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44105
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:22.306245 29175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:22.306511 29175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:22.306577 29175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:22.307757 29175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:22.307803 29175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:22.307833 29175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:22.309289 29175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:22.309536 29175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:22.309810 29175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:22.315533 29181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:22.315454 29182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:22.315469 29185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:22.316061 29175 server_base.cc:1034] running on GCE node
I20260807 22:35:22.316224 29175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:22.316422 29175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:22.317571 29175 hybrid_clock.cc:648] HybridClock initialized: now 1786142122317555 us; error 29 us; skew 500 ppm
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:22.320464 29175 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:22.321147 29175 webserver.cc:460] Webserver started at http://127.28.82.193:42601/ using document root <none> and password file <none>
I20260807 22:35:22.321405 29175 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:22.321477 29175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:22.321636 29175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:22.322748 29175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "efe9582ea8f243ab9f383c7c3523c330"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "f0fc5a63ffb47f3458db41d3294b2f99"
server_key_iv: "208ecd5edb5a2983ee78320290a766f7"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.323125 29175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "efe9582ea8f243ab9f383c7c3523c330"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "f0fc5a63ffb47f3458db41d3294b2f99"
server_key_iv: "208ecd5edb5a2983ee78320290a766f7"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.324638 29175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:22.325428 29199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.325657 29175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:22.325760 29175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "efe9582ea8f243ab9f383c7c3523c330"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "f0fc5a63ffb47f3458db41d3294b2f99"
server_key_iv: "208ecd5edb5a2983ee78320290a766f7"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.325853 29175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:22.337782 29175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:22.339614 29175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:22.339777 29175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:22.340091 29175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:22.340477 29175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:22.340535 29175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.340588 29175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:22.340617 29175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.352476 29175 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:43417
I20260807 22:35:22.352514 29361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:43417 every 8 connection(s)
I20260807 22:35:22.352869 29175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:22.357396 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29175
I20260807 22:35:22.357743 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:22.357928 29003 external_mini_cluster.cc:1428] Setting key dad67049d59e551e72f16bf9036105b3
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:22.362432 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.353643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:41771 (local address 127.28.82.254:44105)
0807 22:35:22.353727 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:22.353729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:22.353931 (+ 202us) server_negotiation.cc:407] Connection header received
0807 22:35:22.354395 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:22.354396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:22.354407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:22.354479 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:22.355540 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.356032 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.356956 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.357133 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.357182 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.359455 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:22.359456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:22.359457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:22.359465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:22.360905 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.361355 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.361357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.361357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.361420 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.361774 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.361776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.361776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.361894 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:22.361968 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:22.362255 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 22:35:22.362341 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:35:22.362628 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.353613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44105 (local address 127.28.82.193:41771)
0807 22:35:22.353809 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:22.353845 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:22.354315 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:22.354579 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:22.354583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:22.355268 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:22.355454 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:22.355458 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.356115 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.356116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:22.356879 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:22.356881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.357204 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.357205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:22.357319 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.357756 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:22.357764 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:22.359353 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:22.361008 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:22.361015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:22.361018 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:22.361273 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:22.361576 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:22.361576 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:22.361577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:22.361702 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:22.361971 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:22.361978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:22.362112 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 22:35:22.362218 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 22:35:22.362373 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260807 22:35:22.363081 29363 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44105
I20260807 22:35:22.363174 29363 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:22.363363 29363 heartbeater.cc:511] Master 127.28.82.254:44105 requested a full tablet report, sending...
I20260807 22:35:22.363929 29077 ts_manager.cc:194] Registered new tserver with Master: efe9582ea8f243ab9f383c7c3523c330 (127.28.82.193:43417)
I20260807 22:35:22.364885 29077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:41771
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:22.385692 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44105
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:22.468479 29371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:22.468688 29371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:22.468712 29371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:22.469775 29371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:22.469815 29371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:22.469841 29371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:22.471311 29371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:22.471508 29371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:22.471688 29371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:22.476999 29380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:22.477015 29383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:22.477020 29379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:22.477037 29371 server_base.cc:1034] running on GCE node
I20260807 22:35:22.477581 29371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:22.477837 29371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:22.479001 29371 hybrid_clock.cc:648] HybridClock initialized: now 1786142122478996 us; error 38 us; skew 500 ppm
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:22.481557 29371 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:22.482055 29371 webserver.cc:460] Webserver started at http://127.28.82.194:37373/ using document root <none> and password file <none>
I20260807 22:35:22.482313 29371 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:22.482386 29371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:22.482563 29371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:22.483628 29371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "4df295aeee28476489dddbdb9bca9b11"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "75c1cc1865c12d7afd3f3ea08d9c11e6"
server_key_iv: "d571a511d821ef6497486b50b5d34d40"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.484006 29371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "4df295aeee28476489dddbdb9bca9b11"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "75c1cc1865c12d7afd3f3ea08d9c11e6"
server_key_iv: "d571a511d821ef6497486b50b5d34d40"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.485383 29371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:22.486296 29391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.486577 29371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:22.486689 29371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "4df295aeee28476489dddbdb9bca9b11"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "75c1cc1865c12d7afd3f3ea08d9c11e6"
server_key_iv: "d571a511d821ef6497486b50b5d34d40"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.486776 29371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:22.495748 29371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:22.497746 29371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:22.497912 29371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:22.498147 29371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:22.498508 29371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:22.498570 29371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.498621 29371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:22.498651 29371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.509649 29371 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:34449
I20260807 22:35:22.509707 29541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:34449 every 8 connection(s)
I20260807 22:35:22.510061 29371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:22.511181 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29371
I20260807 22:35:22.511282 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:22.511433 29003 external_mini_cluster.cc:1428] Setting key 5febe6324feb0750d715148aa7b63bcc
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:22.521369 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.510933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:57411 (local address 127.28.82.254:44105)
0807 22:35:22.511009 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:22.511028 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:22.511262 (+ 234us) server_negotiation.cc:407] Connection header received
0807 22:35:22.511911 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:22.511912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:22.511923 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:22.511987 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:22.513032 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.514156 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.515280 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.515578 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.515637 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.518078 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:22.518080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:22.518080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:22.518089 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:22.519907 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.520328 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.520330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.520330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.520399 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.520662 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.520663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.520663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.520814 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:22.520891 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:22.521169 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 22:35:22.521264 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:35:22.521518 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.510885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44105 (local address 127.28.82.194:57411)
0807 22:35:22.511114 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:22.511171 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:22.511814 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:22.512110 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:22.512115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:22.512730 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:22.512923 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:22.512928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.514315 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.514316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:22.515180 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:22.515183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.515665 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.515671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:22.515824 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.516361 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:22.516372 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:22.517974 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:22.520001 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:22.520007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:22.520009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:22.520235 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:22.520475 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:22.520476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:22.520476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:22.520579 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:22.520885 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:22.520890 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:22.521029 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 22:35:22.521120 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 22:35:22.521233 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260807 22:35:22.522037 29543 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44105
I20260807 22:35:22.522130 29543 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:22.522327 29543 heartbeater.cc:511] Master 127.28.82.254:44105 requested a full tablet report, sending...
I20260807 22:35:22.522795 29077 ts_manager.cc:194] Registered new tserver with Master: 4df295aeee28476489dddbdb9bca9b11 (127.28.82.194:34449)
I20260807 22:35:22.523175 29077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:57411
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:22.543212 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44105
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:22.628154 29548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:22.628357 29548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:22.628381 29548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:22.629606 29548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:22.629653 29548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:22.629681 29548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:22.631203 29548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:22.631450 29548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:22.631687 29548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:22.637164 29557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:22.637131 29556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:22.637136 29560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:22.637446 29548 server_base.cc:1034] running on GCE node
I20260807 22:35:22.637668 29548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:22.637919 29548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:22.639083 29548 hybrid_clock.cc:648] HybridClock initialized: now 1786142122639078 us; error 37 us; skew 500 ppm
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:22.641721 29548 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:22.642268 29548 webserver.cc:460] Webserver started at http://127.28.82.195:46539/ using document root <none> and password file <none>
I20260807 22:35:22.642572 29548 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:22.642645 29548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:22.642812 29548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:22.643865 29548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "26578afc13694acf814037eaf1095258"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "7e1597a3d9233fac2bc3b707a7671a4b"
server_key_iv: "dc384959fefa10ef12c4231d48685b6a"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.644248 29548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "26578afc13694acf814037eaf1095258"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "7e1597a3d9233fac2bc3b707a7671a4b"
server_key_iv: "dc384959fefa10ef12c4231d48685b6a"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.645762 29548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:22.646615 29572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.646817 29548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:22.646924 29548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "26578afc13694acf814037eaf1095258"
format_stamp: "Formatted at 2026-08-07 22:35:22 on dist-test-slave-7kzw"
server_key: "7e1597a3d9233fac2bc3b707a7671a4b"
server_key_iv: "dc384959fefa10ef12c4231d48685b6a"
server_key_version: "encryptionkey@0"
I20260807 22:35:22.647007 29548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:22.672617 29548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:22.674834 29548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:22.674958 29548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:22.675194 29548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:22.675586 29548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:22.675649 29548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.675694 29548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:22.675721 29548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:22.686861 29548 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:45605
I20260807 22:35:22.686906 29735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:45605 every 8 connection(s)
I20260807 22:35:22.687304 29548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:22.690681 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29548
I20260807 22:35:22.690783 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:22.690968 29003 external_mini_cluster.cc:1428] Setting key 543fbd89f309158601e99d2d8d4d3061
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:22.697522 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.688240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:55175 (local address 127.28.82.254:44105)
0807 22:35:22.688294 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:22.688297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:22.688535 (+ 238us) server_negotiation.cc:407] Connection header received
0807 22:35:22.689028 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:22.689029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:22.689040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:22.689094 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:22.690123 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.690688 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.692085 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.692370 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.692455 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.694699 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:22.694701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:22.694701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:22.694724 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:22.696219 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.696597 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.696598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.696598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.696654 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.696874 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.696875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.696875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.696982 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:22.697034 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:22.697388 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 22:35:22.697456 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:22.697718 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.688229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44105 (local address 127.28.82.195:55175)
0807 22:35:22.688407 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:22.688444 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:22.688943 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:22.689149 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:22.689152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:22.689828 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:22.690039 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:22.690044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.690788 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.690789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:22.691999 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:22.692002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.692460 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.692462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:22.692608 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.693098 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:22.693107 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:22.694599 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:22.696294 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:22.696301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:22.696302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:22.696528 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:22.696715 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:22.696716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:22.696716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:22.696817 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:22.697107 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:22.697113 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:22.697261 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 22:35:22.697353 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:35:22.697478 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260807 22:35:22.698257 29736 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44105
I20260807 22:35:22.698345 29736 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:22.698550 29736 heartbeater.cc:511] Master 127.28.82.254:44105 requested a full tablet report, sending...
I20260807 22:35:22.698971 29077 ts_manager.cc:194] Registered new tserver with Master: 26578afc13694acf814037eaf1095258 (127.28.82.195:45605)
I20260807 22:35:22.699368 29077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:55175
I20260807 22:35:22.703405 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:22.729939 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.722019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.28.82.193:43417)
0807 22:35:22.722174 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:22.722181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:22.722199 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:35:22.722226 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:22.722228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:22.722242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:22.722310 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:22.722928 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.723782 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.724513 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.724834 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.724903 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.727205 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:22.727212 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:22.727213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:22.727219 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:22.728821 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.729175 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.729178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.729178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.729237 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.729454 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.729455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.729455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.729594 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:22.729716 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:22.729790 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 22:35:22.729856 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
W20260807 22:35:22.730336 29242 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47528
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:22.755756 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:22.748780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47530 (local address 127.28.82.193:43417)
0807 22:35:22.748845 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:22.748847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:22.748877 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:35:22.748981 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:22.748982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:22.748995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:22.749068 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:22.749721 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.750233 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.751079 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:22.751295 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:22.751405 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:22.753589 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:22.753590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:22.753590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:22.753597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:22.754851 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.755152 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.755152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.755153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.755213 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:22.755380 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:22.755381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:22.755381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:22.755467 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:22.755570 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:22.755624 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 22:35:22.755688 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:35:22.756476 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29175
I20260807 22:35:22.765321 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29371
I20260807 22:35:22.776513 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29548
I20260807 22:35:22.785902 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29032
2026-08-07T22:35:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2198 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29759](info): set up 2 sockets
Aug 07 22:35:22 dist-test-slave-7kzw krb5kdc[29759](info): commencing operation
krb5kdc: starting...
W20260807 22:35:24.123669 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:24.249631 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:33257
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:33257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:24.332513 29778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:24.332726 29778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:24.332749 29778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:24.333878 29778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:24.333920 29778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:24.333937 29778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:24.333950 29778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:24.334002 29778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:24.335455 29778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:33257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:33257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:24.335716 29778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:24.335943 29778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:24.341210 29790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:24.341166 29787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:24.341166 29784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:24.341287 29778 server_base.cc:1034] running on GCE node
I20260807 22:35:24.341645 29778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:24.341893 29778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:24.343046 29778 hybrid_clock.cc:648] HybridClock initialized: now 1786142124343033 us; error 28 us; skew 500 ppm
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:24.350003 29778 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:24.350579 29778 webserver.cc:460] Webserver started at http://127.28.82.254:41303/ using document root <none> and password file <none>
I20260807 22:35:24.350805 29778 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:24.350872 29778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:24.351032 29778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:24.352092 29778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "4047ec055d934914a48de79e97148134"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "842e7638ed4d2ed171aa87ee26854c88"
server_key_iv: "2d1ff9be55234c5754bd50bf0c044349"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.352476 29778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "4047ec055d934914a48de79e97148134"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "842e7638ed4d2ed171aa87ee26854c88"
server_key_iv: "2d1ff9be55234c5754bd50bf0c044349"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.353899 29778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:24.354733 29801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.354964 29778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:24.355073 29778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "4047ec055d934914a48de79e97148134"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "842e7638ed4d2ed171aa87ee26854c88"
server_key_iv: "2d1ff9be55234c5754bd50bf0c044349"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.355147 29778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:24.382068 29778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:24.384346 29778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:24.384516 29778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:24.389218 29778 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:33257
I20260807 22:35:24.389271 29869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:33257 every 8 connection(s)
I20260807 22:35:24.389868 29778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:24.390671 29870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:24.393555 29870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134: Bootstrap starting.
I20260807 22:35:24.394099 29870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:24.394352 29870 log.cc:824] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:24.395068 29870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134: No bootstrap required, opened a new log
I20260807 22:35:24.395123 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29778
I20260807 22:35:24.395231 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:24.395394 29003 external_mini_cluster.cc:1428] Setting key ae045c12c76704fb5b80adc40caf66a2
I20260807 22:35:24.398000 29870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4047ec055d934914a48de79e97148134" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33257 } }
I20260807 22:35:24.398170 29870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:24.398207 29870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4047ec055d934914a48de79e97148134, State: Initialized, Role: FOLLOWER
I20260807 22:35:24.398326 29870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4047ec055d934914a48de79e97148134" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33257 } }
I20260807 22:35:24.398409 29870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:24.398438 29870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:24.398507 29870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:24.399166 29870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4047ec055d934914a48de79e97148134" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33257 } }
I20260807 22:35:24.399272 29870 leader_election.cc:302] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4047ec055d934914a48de79e97148134; no voters:
I20260807 22:35:24.399443 29870 leader_election.cc:288] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:24.399464 29877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:24.399590 29877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [term 1 LEADER]: Becoming Leader. State: Replica: 4047ec055d934914a48de79e97148134, State: Running, Role: LEADER
I20260807 22:35:24.399690 29877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4047ec055d934914a48de79e97148134" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33257 } }
I20260807 22:35:24.399732 29870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:24.400031 29879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4047ec055d934914a48de79e97148134. Latest consensus state: current_term: 1 leader_uuid: "4047ec055d934914a48de79e97148134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4047ec055d934914a48de79e97148134" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33257 } } }
I20260807 22:35:24.400122 29879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:24.400286 29878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4047ec055d934914a48de79e97148134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4047ec055d934914a48de79e97148134" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33257 } } }
I20260807 22:35:24.400363 29878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134 [sys.catalog]: This master's current role is: LEADER
W20260807 22:35:24.401794 29899 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:35:24.401854 29899 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:35:24.401916 29883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:24.402104 29883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:24.404559 29883 catalog_manager.cc:1347] Generated new cluster ID: 626202e12f8544cd80a36de597af7e8b
I20260807 22:35:24.404615 29883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:24.407917 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.395963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.28.82.254:33257)
0807 22:35:24.396177 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:24.396187 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:24.396222 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:35:24.397035 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:24.397044 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:24.397109 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:24.397276 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:24.397869 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.399282 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.400392 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.400849 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.400909 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.403918 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:24.403922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:24.403922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:24.403934 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:24.406043 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.406445 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.406448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.406448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.406584 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.406813 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.406813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.406814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.407123 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:24.407463 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:24.407542 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 22:35:24.407688 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260807 22:35:24.422065 29883 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:24.422657 29883 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:24.427469 29883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4047ec055d934914a48de79e97148134: Generated new TSK 0
I20260807 22:35:24.427657 29883 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:24.463690 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33257
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:24.547350 29905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:24.547552 29905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:24.547576 29905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:24.548731 29905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:24.548774 29905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:24.548800 29905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:24.550202 29905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33257
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:24.550380 29905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:24.550577 29905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:24.555567 29917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:24.555634 29915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:24.555760 29905 server_base.cc:1034] running on GCE node
W20260807 22:35:24.555573 29914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:24.556033 29905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:24.556277 29905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:24.557442 29905 hybrid_clock.cc:648] HybridClock initialized: now 1786142124557439 us; error 37 us; skew 500 ppm
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:24.560237 29905 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:24.560668 29905 webserver.cc:460] Webserver started at http://127.28.82.193:40273/ using document root <none> and password file <none>
I20260807 22:35:24.560886 29905 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:24.560953 29905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:24.561101 29905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:24.562155 29905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "80b1b007bc144467a3275a2aa0388fd6"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "d08bba84e602f624398c8629281d2507"
server_key_iv: "d6e303167c93999152fa6043e6f58138"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.562588 29905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "80b1b007bc144467a3275a2aa0388fd6"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "d08bba84e602f624398c8629281d2507"
server_key_iv: "d6e303167c93999152fa6043e6f58138"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.563939 29905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:24.564646 29927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.564812 29905 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:24.564914 29905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "80b1b007bc144467a3275a2aa0388fd6"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "d08bba84e602f624398c8629281d2507"
server_key_iv: "d6e303167c93999152fa6043e6f58138"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.564994 29905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:24.586081 29905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:24.588080 29905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:24.588253 29905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:24.588491 29905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:24.588850 29905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:24.588912 29905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.588953 29905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:24.588982 29905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.594895 29905 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:35205
I20260807 22:35:24.594947 30097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:35205 every 8 connection(s)
I20260807 22:35:24.595259 29905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:24.599069 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29905
I20260807 22:35:24.599174 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:24.599336 29003 external_mini_cluster.cc:1428] Setting key faa190aecc28dc0e13a6ac0302370f2d
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:24.606124 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.595961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:33257 (local address 127.28.82.193:41559)
0807 22:35:24.596121 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:24.596164 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:24.597153 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:24.597374 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:24.597378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:24.597625 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:24.597815 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.597819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.598624 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.598625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.599840 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.599843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.600313 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.600314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.600492 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.601027 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:24.601038 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:24.602663 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:24.604692 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:24.604699 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:24.604700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:24.604920 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:24.605201 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:24.605202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:24.605202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:24.605305 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:24.605629 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:24.605633 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:24.605767 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 22:35:24.605850 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:35:24.605967 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260807 22:35:24.606173 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.596019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:41559 (local address 127.28.82.254:33257)
0807 22:35:24.596102 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:24.596104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:24.596256 (+ 152us) server_negotiation.cc:407] Connection header received
0807 22:35:24.597253 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:24.597254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:24.597264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:24.597320 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:24.597926 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.598426 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.599915 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.600207 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.600294 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.602777 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:24.602779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:24.602779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:24.602789 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:24.604590 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.605031 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.605032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.605033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.605113 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.605382 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.605383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.605384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.605557 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:24.605645 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:24.605916 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 22:35:24.606023 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:35:24.606684 30098 heartbeater.cc:347] Connected to a master server at 127.28.82.254:33257
I20260807 22:35:24.606778 30098 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:24.606956 30098 heartbeater.cc:511] Master 127.28.82.254:33257 requested a full tablet report, sending...
I20260807 22:35:24.607493 29821 ts_manager.cc:194] Registered new tserver with Master: 80b1b007bc144467a3275a2aa0388fd6 (127.28.82.193:35205)
I20260807 22:35:24.608100 29821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:41559
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:24.627523 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33257
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:24.709849 30104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:24.710098 30104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:24.710136 30104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:24.711301 30104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:24.711367 30104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:24.711413 30104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:24.712881 30104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33257
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:24.713141 30104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:24.713387 30104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:24.718401 30115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:24.718396 30114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:24.718395 30117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:24.718644 30104 server_base.cc:1034] running on GCE node
I20260807 22:35:24.718885 30104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:24.719116 30104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:24.720294 30104 hybrid_clock.cc:648] HybridClock initialized: now 1786142124720273 us; error 37 us; skew 500 ppm
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:24.722874 30104 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:24.723249 30104 webserver.cc:460] Webserver started at http://127.28.82.194:44043/ using document root <none> and password file <none>
I20260807 22:35:24.723423 30104 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:24.723464 30104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:24.723551 30104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:24.724535 30104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "0a9c1ed8567546d1841ddb9b17e13345"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "28990b707893ff7f6e01a2b75a4c1f13"
server_key_iv: "33084b6ef9fa820d850fae743e8471a7"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.724848 30104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "0a9c1ed8567546d1841ddb9b17e13345"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "28990b707893ff7f6e01a2b75a4c1f13"
server_key_iv: "33084b6ef9fa820d850fae743e8471a7"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.726106 30104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:24.726861 30131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.727047 30104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:35:24.727133 30104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "0a9c1ed8567546d1841ddb9b17e13345"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "28990b707893ff7f6e01a2b75a4c1f13"
server_key_iv: "33084b6ef9fa820d850fae743e8471a7"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.727190 30104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:24.746541 30104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:24.748412 30104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:24.748523 30104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:24.748708 30104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:24.749022 30104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:24.749054 30104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.749097 30104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:24.749123 30104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.754925 30104 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:44963
I20260807 22:35:24.754985 30296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:44963 every 8 connection(s)
I20260807 22:35:24.755288 30104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:24.762915 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 30104
I20260807 22:35:24.763012 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:24.763217 29003 external_mini_cluster.cc:1428] Setting key 02b3215a52b9d555442b889d70663539
I20260807 22:35:24.765144 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.756005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:59645 (local address 127.28.82.254:33257)
0807 22:35:24.756059 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:24.756062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:24.756300 (+ 238us) server_negotiation.cc:407] Connection header received
0807 22:35:24.757290 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:24.757291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:24.757304 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:24.757392 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:24.758084 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.758708 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.759688 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.759868 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.759919 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.761997 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:24.761998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:24.761999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:24.762008 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:24.763524 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.763999 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.764000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.764000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.764074 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.764321 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.764322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.764322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.764490 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:24.764565 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:24.764965 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 22:35:24.765053 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:35:24.765655 30300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.755995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:33257 (local address 127.28.82.194:59645)
0807 22:35:24.756173 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:24.756208 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:24.757165 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:24.757494 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:24.757499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:24.757819 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:24.758007 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.758011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.758820 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.758821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.759613 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.759615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.759940 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.759941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.760034 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.760462 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:24.760470 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:24.761898 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:24.763611 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:24.763618 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:24.763620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:24.763920 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:24.764146 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:24.764146 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:24.764147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:24.764250 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:24.764591 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:24.764599 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:24.764778 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 22:35:24.765241 (+ 463us) client_negotiation.cc:240] Negotiation successful
0807 22:35:24.765413 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260807 22:35:24.766877 30297 heartbeater.cc:347] Connected to a master server at 127.28.82.254:33257
I20260807 22:35:24.766969 30297 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:24.767146 30297 heartbeater.cc:511] Master 127.28.82.254:33257 requested a full tablet report, sending...
I20260807 22:35:24.767592 29821 ts_manager.cc:194] Registered new tserver with Master: 0a9c1ed8567546d1841ddb9b17e13345 (127.28.82.194:44963)
I20260807 22:35:24.768100 29821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:59645
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:24.791390 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33257
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:24.874351 30304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:24.874574 30304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:24.874599 30304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:24.875710 30304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:24.875765 30304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:24.875808 30304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:24.877242 30304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33257
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:24.877427 30304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:24.877597 30304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:24.882545 30317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:24.882625 30316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:24.882731 30304 server_base.cc:1034] running on GCE node
W20260807 22:35:24.882552 30321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:24.883109 30304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:24.883309 30304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:24.884501 30304 hybrid_clock.cc:648] HybridClock initialized: now 1786142124884483 us; error 52 us; skew 500 ppm
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:24.887368 30304 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:24.887862 30304 webserver.cc:460] Webserver started at http://127.28.82.195:40621/ using document root <none> and password file <none>
I20260807 22:35:24.888092 30304 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:24.888159 30304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:24.888310 30304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:24.889325 30304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "b00a44b01309484b8e9ebb8d2d2a400f"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "9220bdc24ca265e805c34d5e4a3ee42d"
server_key_iv: "736a775c9b7bbc948a5e5d36fe1ff00a"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.889701 30304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "b00a44b01309484b8e9ebb8d2d2a400f"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "9220bdc24ca265e805c34d5e4a3ee42d"
server_key_iv: "736a775c9b7bbc948a5e5d36fe1ff00a"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.891115 30304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 22:35:24.891850 30332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.892027 30304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:24.892133 30304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "b00a44b01309484b8e9ebb8d2d2a400f"
format_stamp: "Formatted at 2026-08-07 22:35:24 on dist-test-slave-7kzw"
server_key: "9220bdc24ca265e805c34d5e4a3ee42d"
server_key_iv: "736a775c9b7bbc948a5e5d36fe1ff00a"
server_key_version: "encryptionkey@0"
I20260807 22:35:24.892206 30304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:24.905217 30304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:24.907538 30304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:24.907698 30304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:24.907922 30304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:24.908236 30304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:24.908295 30304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.908337 30304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:24.908363 30304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:24.914410 30304 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:46273
I20260807 22:35:24.914487 30509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:46273 every 8 connection(s)
I20260807 22:35:24.914824 30304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:24.916370 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 30304
I20260807 22:35:24.916474 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:24.916646 29003 external_mini_cluster.cc:1428] Setting key b80a97e866884fc22fe967746014ce07
Aug 07 22:35:24 dist-test-slave-7kzw krb5kdc[29759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:24.924279 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.915570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:50143 (local address 127.28.82.254:33257)
0807 22:35:24.915615 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:24.915618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:24.915853 (+ 235us) server_negotiation.cc:407] Connection header received
0807 22:35:24.916863 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:24.916865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:24.916879 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:24.916936 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:24.917607 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.918121 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.919205 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.919406 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.919491 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.921520 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:24.921522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:24.921522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:24.921530 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:24.922862 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.923287 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.923288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.923289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.923364 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.923633 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.923634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.923634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.923750 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:24.923825 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:24.924109 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 22:35:24.924189 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 22:35:24.924454 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.915563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:33257 (local address 127.28.82.195:50143)
0807 22:35:24.915728 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:24.915763 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:24.916746 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:24.917068 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:24.917071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:24.917324 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:24.917501 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.917505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.918220 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.918221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.919103 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.919106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.919502 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.919503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.919630 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.920071 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:24.920080 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:24.921422 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:24.922946 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:24.922952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:24.922954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:24.923207 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:24.923439 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:24.923440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:24.923441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:24.923549 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:24.923827 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:24.923834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:24.923968 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 22:35:24.924079 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 22:35:24.924230 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260807 22:35:24.924999 30510 heartbeater.cc:347] Connected to a master server at 127.28.82.254:33257
I20260807 22:35:24.925081 30510 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:24.925277 30510 heartbeater.cc:511] Master 127.28.82.254:33257 requested a full tablet report, sending...
I20260807 22:35:24.925663 29821 ts_manager.cc:194] Registered new tserver with Master: b00a44b01309484b8e9ebb8d2d2a400f (127.28.82.195:46273)
I20260807 22:35:24.926025 29821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:50143
I20260807 22:35:24.928202 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:35:24.935413 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.929764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.28.82.254:33257)
0807 22:35:24.929810 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:24.929812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:24.929901 (+ 89us) server_negotiation.cc:407] Connection header received
0807 22:35:24.929936 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:24.929937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:24.929945 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:24.929991 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:24.930514 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.931002 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.931678 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.931850 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.931897 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.932974 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:24.932976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:24.932976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:24.932983 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:24.934334 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.934774 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.934775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.934775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.934833 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:24.934995 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:24.934995 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:24.934995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:24.935132 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:24.935205 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:24.935294 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 22:35:24.935353 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:35:24.937506 29821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:24.939194 29821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:24.946302 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.942052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59032 (local address 127.28.82.195:46273)
0807 22:35:24.942245 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:24.942252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:24.942661 (+ 409us) server_negotiation.cc:407] Connection header received
0807 22:35:24.942894 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:24.942897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:24.942912 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:24.942993 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:24.943107 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.944034 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.945132 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.945883 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.945965 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.946023 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:24.946186 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 22:35:24.946225 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260807 22:35:24.946337 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.941827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:46273 (local address 127.0.0.1:59032)
0807 22:35:24.942417 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:24.942423 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:24.942822 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:24.943003 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:24.943003 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:24.943005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:24.943049 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.943051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.944142 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.944143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.945042 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.945045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.945978 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.945979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.946069 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.946082 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:35:24.946158 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 22:35:24.946246 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260807 22:35:24.946673 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.941932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.28.82.193:35205)
0807 22:35:24.942308 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:24.942312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:24.942322 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:24.942366 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:24.942369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:24.942386 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:24.942503 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:24.942660 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.943742 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.945222 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.945896 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.945965 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.946023 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:24.946536 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 22:35:24.946581 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":81,"threads_started":1}
I20260807 22:35:24.947229 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.941811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:44963 (local address 127.0.0.1:32854)
0807 22:35:24.942325 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:24.942332 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:24.942366 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:24.942700 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:24.942700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:24.942702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:24.942750 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.942752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.943738 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.943739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.945140 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.945142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.947005 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.947006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.947072 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.947075 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:24.947127 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 22:35:24.947164 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":38,"thread_start_us":76,"threads_started":1}
I20260807 22:35:24.947275 30402 tablet_service.cc:1467] Processing CreateTablet for tablet fe2808a2122c46a08997cb0de36bb191 (DEFAULT_TABLE table=test-table [id=4934557b3d2d4658947ed8719593f4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:24.947561 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.942062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.28.82.194:44963)
0807 22:35:24.942330 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:24.942334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:24.942366 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:35:24.942419 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:24.942421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:24.942434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:24.942544 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:24.942841 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.943630 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.946064 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.946838 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.946913 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.946968 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:24.947444 (+ 476us) server_negotiation.cc:299] Negotiation successful
0807 22:35:24.947489 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":110,"threads_started":1}
I20260807 22:35:24.947628 30402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2808a2122c46a08997cb0de36bb191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:24.948124 30198 tablet_service.cc:1467] Processing CreateTablet for tablet fe2808a2122c46a08997cb0de36bb191 (DEFAULT_TABLE table=test-table [id=4934557b3d2d4658947ed8719593f4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:24.948436 30198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2808a2122c46a08997cb0de36bb191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:24.950409 30531 tablet_bootstrap.cc:492] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f: Bootstrap starting.
I20260807 22:35:24.946337 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:24.941803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:35205 (local address 127.0.0.1:55686)
0807 22:35:24.942124 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:24.942166 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:24.942230 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:24.942518 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:24.942521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:24.942523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:24.942591 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.942593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.943841 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.943841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.945137 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:24.945140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:24.945976 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:24.945978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:24.946068 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:24.946082 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:35:24.946158 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 22:35:24.946246 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":96,"thread_start_us":56,"threads_started":1}
I20260807 22:35:24.951150 30531 tablet_bootstrap.cc:654] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:24.951176 30532 tablet_bootstrap.cc:492] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345: Bootstrap starting.
I20260807 22:35:24.951323 29995 tablet_service.cc:1467] Processing CreateTablet for tablet fe2808a2122c46a08997cb0de36bb191 (DEFAULT_TABLE table=test-table [id=4934557b3d2d4658947ed8719593f4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:24.951702 29995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe2808a2122c46a08997cb0de36bb191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:24.951772 30531 log.cc:824] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:24.951931 30532 tablet_bootstrap.cc:654] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:24.952494 30532 log.cc:824] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:24.952667 30531 tablet_bootstrap.cc:492] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f: No bootstrap required, opened a new log
I20260807 22:35:24.952754 30531 ts_tablet_manager.cc:1397] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:35:24.954873 30531 raft_consensus.cc:348] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:24.955054 30531 raft_consensus.cc:374] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:24.955057 30537 tablet_bootstrap.cc:492] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6: Bootstrap starting.
I20260807 22:35:24.955092 30531 raft_consensus.cc:729] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b00a44b01309484b8e9ebb8d2d2a400f, State: Initialized, Role: FOLLOWER
I20260807 22:35:24.955204 30531 consensus_queue.cc:260] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:24.955498 30531 ts_tablet_manager.cc:1428] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:24.955521 30510 heartbeater.cc:503] Master 127.28.82.254:33257 was elected leader, sending a full tablet report...
I20260807 22:35:24.955721 30537 tablet_bootstrap.cc:654] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:24.956039 30532 tablet_bootstrap.cc:492] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345: No bootstrap required, opened a new log
I20260807 22:35:24.956089 30532 ts_tablet_manager.cc:1397] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 22:35:24.956264 30537 log.cc:824] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:24.957576 30532 raft_consensus.cc:348] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:24.957753 30532 raft_consensus.cc:374] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:24.957782 30532 raft_consensus.cc:729] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a9c1ed8567546d1841ddb9b17e13345, State: Initialized, Role: FOLLOWER
I20260807 22:35:24.957894 30532 consensus_queue.cc:260] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:24.958124 30532 ts_tablet_manager.cc:1428] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:35:24.958207 30297 heartbeater.cc:503] Master 127.28.82.254:33257 was elected leader, sending a full tablet report...
I20260807 22:35:24.958884 30537 tablet_bootstrap.cc:492] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6: No bootstrap required, opened a new log
I20260807 22:35:24.958976 30537 ts_tablet_manager.cc:1397] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 22:35:24.960352 30537 raft_consensus.cc:348] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:24.960481 30537 raft_consensus.cc:374] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:24.960503 30537 raft_consensus.cc:729] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b1b007bc144467a3275a2aa0388fd6, State: Initialized, Role: FOLLOWER
I20260807 22:35:24.960610 30537 consensus_queue.cc:260] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:24.960866 30537 ts_tablet_manager.cc:1428] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:35:24.960904 30098 heartbeater.cc:503] Master 127.28.82.254:33257 was elected leader, sending a full tablet report...
W20260807 22:35:25.005823 30299 tablet.cc:2369] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:35:25.095981 30099 tablet.cc:2369] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:35:25.165437 30511 tablet.cc:2369] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:25.245688 30541 raft_consensus.cc:482] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:25.245827 30541 raft_consensus.cc:504] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:25.246177 30541 leader_election.cc:288] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b00a44b01309484b8e9ebb8d2d2a400f (127.28.82.195:46273), 0a9c1ed8567546d1841ddb9b17e13345 (127.28.82.194:44963)
I20260807 22:35:25.249084 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:25.246457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:60273 (local address 127.28.82.194:44963)
0807 22:35:25.246527 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:25.246530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:25.246599 (+ 69us) server_negotiation.cc:407] Connection header received
0807 22:35:25.246664 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:25.246665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:25.246675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:25.246748 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:25.246911 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.247398 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.248369 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.248733 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.248817 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:25.248826 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:25.248983 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 22:35:25.249019 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:35:25.249096 30542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:25.246356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:44963 (local address 127.28.82.193:60273)
0807 22:35:25.246524 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:25.246537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:25.246604 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:25.246755 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:25.246756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:25.246759 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:25.246841 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:25.246842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.247482 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.247483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:25.248276 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:25.248278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.248823 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.248824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:25.248902 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:25.248905 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:25.248964 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:35:25.249005 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":5,"thread_start_us":66,"threads_started":1}
I20260807 22:35:25.249179 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:25.246362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:46273 (local address 127.28.82.193:34695)
0807 22:35:25.246605 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:25.246616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:25.246703 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:25.246876 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:25.246877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:25.246879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:25.246970 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:25.246971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.247601 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.247603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:25.248347 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:25.248348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.248919 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.248920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:25.249000 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:25.249004 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:25.249063 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:35:25.249104 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":68,"threads_started":1}
I20260807 22:35:25.249186 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:25.246422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:34695 (local address 127.28.82.195:46273)
0807 22:35:25.246515 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:25.246518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:25.246671 (+ 153us) server_negotiation.cc:407] Connection header received
0807 22:35:25.246788 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:25.246789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:25.246801 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:25.246871 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:25.247041 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.247513 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.248442 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.248832 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.248919 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:25.248928 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:25.249091 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 22:35:25.249128 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:35:25.249424 30234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2808a2122c46a08997cb0de36bb191" candidate_uuid: "80b1b007bc144467a3275a2aa0388fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a9c1ed8567546d1841ddb9b17e13345" is_pre_election: true
I20260807 22:35:25.249588 30234 raft_consensus.cc:2454] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b1b007bc144467a3275a2aa0388fd6 in term 0.
I20260807 22:35:25.249578 30432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2808a2122c46a08997cb0de36bb191" candidate_uuid: "80b1b007bc144467a3275a2aa0388fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" is_pre_election: true
I20260807 22:35:25.249752 30432 raft_consensus.cc:2454] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80b1b007bc144467a3275a2aa0388fd6 in term 0.
I20260807 22:35:25.249842 29929 leader_election.cc:302] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a9c1ed8567546d1841ddb9b17e13345, 80b1b007bc144467a3275a2aa0388fd6; no voters:
I20260807 22:35:25.249984 30541 raft_consensus.cc:2781] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:25.250046 30541 raft_consensus.cc:482] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:25.250073 30541 raft_consensus.cc:3037] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:25.250823 30541 raft_consensus.cc:504] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:25.251030 30541 leader_election.cc:288] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [CANDIDATE]: Term 1 election: Requested vote from peers b00a44b01309484b8e9ebb8d2d2a400f (127.28.82.195:46273), 0a9c1ed8567546d1841ddb9b17e13345 (127.28.82.194:44963)
I20260807 22:35:25.251103 30234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2808a2122c46a08997cb0de36bb191" candidate_uuid: "80b1b007bc144467a3275a2aa0388fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a9c1ed8567546d1841ddb9b17e13345"
I20260807 22:35:25.251147 30432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe2808a2122c46a08997cb0de36bb191" candidate_uuid: "80b1b007bc144467a3275a2aa0388fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b00a44b01309484b8e9ebb8d2d2a400f"
I20260807 22:35:25.251207 30432 raft_consensus.cc:3037] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:25.251216 30234 raft_consensus.cc:3037] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:25.251897 30432 raft_consensus.cc:2454] T fe2808a2122c46a08997cb0de36bb191 P b00a44b01309484b8e9ebb8d2d2a400f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b1b007bc144467a3275a2aa0388fd6 in term 1.
I20260807 22:35:25.252009 30234 raft_consensus.cc:2454] T fe2808a2122c46a08997cb0de36bb191 P 0a9c1ed8567546d1841ddb9b17e13345 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80b1b007bc144467a3275a2aa0388fd6 in term 1.
I20260807 22:35:25.252072 29928 leader_election.cc:302] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80b1b007bc144467a3275a2aa0388fd6, b00a44b01309484b8e9ebb8d2d2a400f; no voters:
I20260807 22:35:25.252177 30541 raft_consensus.cc:2781] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:25.252322 30541 raft_consensus.cc:686] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [term 1 LEADER]: Becoming Leader. State: Replica: 80b1b007bc144467a3275a2aa0388fd6, State: Running, Role: LEADER
I20260807 22:35:25.252413 30541 consensus_queue.cc:237] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } }
I20260807 22:35:25.253248 29821 catalog_manager.cc:5495] T fe2808a2122c46a08997cb0de36bb191 P 80b1b007bc144467a3275a2aa0388fd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80b1b007bc144467a3275a2aa0388fd6 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "80b1b007bc144467a3275a2aa0388fd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b1b007bc144467a3275a2aa0388fd6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 35205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b00a44b01309484b8e9ebb8d2d2a400f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a9c1ed8567546d1841ddb9b17e13345" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44963 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:35:25 dist-test-slave-7kzw krb5kdc[29759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:25.301997 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:25.293947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55702 (local address 127.28.82.193:35205)
0807 22:35:25.294017 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:25.294020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:25.294276 (+ 256us) server_negotiation.cc:407] Connection header received
0807 22:35:25.294350 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:25.294351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:25.294365 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:25.294408 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:25.295025 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.295565 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.296318 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.296499 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.296553 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:25.298982 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:25.298993 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:25.298993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:25.299000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:25.300825 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:25.301214 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:25.301217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:25.301217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:25.301275 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:25.301447 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:25.301448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:25.301448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:25.301644 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:25.301756 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:25.301819 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 22:35:25.301884 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 22:35:25 dist-test-slave-7kzw krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142125, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:35:25 dist-test-slave-7kzw krb5kdc[29759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142125, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:25.326879 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:25.319545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55706 (local address 127.28.82.193:35205)
0807 22:35:25.319618 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:25.319626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:25.319767 (+ 141us) server_negotiation.cc:407] Connection header received
0807 22:35:25.319858 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:25.319860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:25.319875 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:25.319937 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:25.320500 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.320966 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.321688 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.321873 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.321944 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:25.324308 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:25.324310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:25.324310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:25.324322 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:25.325606 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:25.325978 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:25.325980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:25.325980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:25.326041 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:25.326241 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:25.326242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:25.326243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:25.326350 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:25.326500 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:25.326671 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 22:35:25.326769 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 22:35:25 dist-test-slave-7kzw krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:35:25 dist-test-slave-7kzw krb5kdc[29759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:25.351323 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:25.344212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.28.82.193:35205)
0807 22:35:25.344340 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:25.344342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:25.344381 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:35:25.344430 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:25.344431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:25.344440 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:25.344497 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:25.345101 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.345561 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.346309 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:25.346702 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:25.346763 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:25.348981 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:25.348983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:25.348983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:25.348991 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:25.350154 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:25.350554 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:25.350555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:25.350556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:25.350616 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:25.350810 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:25.350811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:25.350812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:25.350913 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:25.351035 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:25.351131 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 22:35:25.351213 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 22:35:25.352002 29968 tablet_service.cc:2908] Rejecting scan request for tablet fe2808a2122c46a08997cb0de36bb191: Uninitialized: clean time has not yet been initialized
I20260807 22:35:25.352970 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29905
I20260807 22:35:25.369417 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 30104
I20260807 22:35:25.375943 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 30304
I20260807 22:35:25.382421 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29778
2026-08-07T22:35:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2591 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:25 dist-test-slave-7kzw krb5kdc[30562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:25 dist-test-slave-7kzw krb5kdc[30562](info): set up 2 sockets
Aug 07 22:35:25 dist-test-slave-7kzw krb5kdc[30562](info): commencing operation
krb5kdc: starting...
W20260807 22:35:26.600579 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:26 dist-test-slave-7kzw krb5kdc[30562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:26.721886 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43905
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:43905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:26.809021 30582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:26.809226 30582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:26.809247 30582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:26.810299 30582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:26.810333 30582 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:35:26.810345 30582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:26.810359 30582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:26.810372 30582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:26.810384 30582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:26.811794 30582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:43905
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:26.811975 30582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:26.812150 30582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:26.816946 30589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:26.816964 30590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:26.816964 30593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:26.817296 30582 server_base.cc:1034] running on GCE node
I20260807 22:35:26.817559 30582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:26.817843 30582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:26.819002 30582 hybrid_clock.cc:648] HybridClock initialized: now 1786142126818978 us; error 63 us; skew 500 ppm
Aug 07 22:35:26 dist-test-slave-7kzw krb5kdc[30562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142126, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:26.821483 30582 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:26.821909 30582 webserver.cc:460] Webserver started at http://127.28.82.254:43221/ using document root <none> and password file <none>
I20260807 22:35:26.822156 30582 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:26.822222 30582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:26.822362 30582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:26.823455 30582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "46ff06a45c2d4be18fdff1aac3527f99"
format_stamp: "Formatted at 2026-08-07 22:35:26 on dist-test-slave-7kzw"
server_key: "20f1f83fe65da90a3cbded410e02a08c"
server_key_iv: "b29ddca46e49698b602f253c2d6eb349"
server_key_version: "encryptionkey@0"
I20260807 22:35:26.823839 30582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "46ff06a45c2d4be18fdff1aac3527f99"
format_stamp: "Formatted at 2026-08-07 22:35:26 on dist-test-slave-7kzw"
server_key: "20f1f83fe65da90a3cbded410e02a08c"
server_key_iv: "b29ddca46e49698b602f253c2d6eb349"
server_key_version: "encryptionkey@0"
I20260807 22:35:26.825285 30582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:26.826098 30603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:26.826305 30582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:26.826421 30582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "46ff06a45c2d4be18fdff1aac3527f99"
format_stamp: "Formatted at 2026-08-07 22:35:26 on dist-test-slave-7kzw"
server_key: "20f1f83fe65da90a3cbded410e02a08c"
server_key_iv: "b29ddca46e49698b602f253c2d6eb349"
server_key_version: "encryptionkey@0"
I20260807 22:35:26.826521 30582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:26.833956 30582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:26.836076 30582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:26.836233 30582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:26.841120 30582 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:43905
I20260807 22:35:26.841251 30694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:43905 every 8 connection(s)
I20260807 22:35:26.841553 30582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:26.842232 30695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:26.844697 30695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99: Bootstrap starting.
I20260807 22:35:26.845330 30695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:26.845642 30695 log.cc:824] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:26.846377 30695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99: No bootstrap required, opened a new log
I20260807 22:35:26.846702 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 30582
I20260807 22:35:26.846818 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:26.847004 29003 external_mini_cluster.cc:1428] Setting key 0adbd215cc7783201697c76b24288aa6
I20260807 22:35:26.849342 30695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ff06a45c2d4be18fdff1aac3527f99" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43905 } }
I20260807 22:35:26.849481 30695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:26.849506 30695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ff06a45c2d4be18fdff1aac3527f99, State: Initialized, Role: FOLLOWER
I20260807 22:35:26.849658 30695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ff06a45c2d4be18fdff1aac3527f99" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43905 } }
I20260807 22:35:26.849732 30695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:26.849773 30695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:26.849874 30695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:26.850539 30695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ff06a45c2d4be18fdff1aac3527f99" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43905 } }
I20260807 22:35:26.850672 30695 leader_election.cc:302] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46ff06a45c2d4be18fdff1aac3527f99; no voters:
I20260807 22:35:26.850858 30695 leader_election.cc:288] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:26.850894 30702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:26.851027 30702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [term 1 LEADER]: Becoming Leader. State: Replica: 46ff06a45c2d4be18fdff1aac3527f99, State: Running, Role: LEADER
I20260807 22:35:26.851187 30695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:26.851181 30702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ff06a45c2d4be18fdff1aac3527f99" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43905 } }
I20260807 22:35:26.853379 30703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46ff06a45c2d4be18fdff1aac3527f99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ff06a45c2d4be18fdff1aac3527f99" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43905 } } }
I20260807 22:35:26.853386 30705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ff06a45c2d4be18fdff1aac3527f99. Latest consensus state: current_term: 1 leader_uuid: "46ff06a45c2d4be18fdff1aac3527f99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ff06a45c2d4be18fdff1aac3527f99" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43905 } } }
I20260807 22:35:26.853569 30703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:26.853583 30705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:26.853976 30719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:26.854483 30719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 22:35:26 dist-test-slave-7kzw krb5kdc[30562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:26.856501 30719 catalog_manager.cc:1347] Generated new cluster ID: acc503c12a8b445da603612f1bafe6bd
I20260807 22:35:26.856568 30719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:26.859396 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:26.847552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.28.82.254:43905)
0807 22:35:26.847762 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:26.847773 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:26.847814 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:35:26.848880 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:26.848890 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:26.848954 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:26.849122 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:26.849839 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:26.851165 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:26.852105 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:26.852517 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:26.852651 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:26.855706 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:26.855710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:26.855710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:26.855723 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:26.857655 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:26.858100 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:26.858104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:26.858104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:26.858206 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:26.858432 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:26.858433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:26.858434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:26.858747 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:26.858944 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:26.859028 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 22:35:26.859172 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":105,"threads_started":1}
I20260807 22:35:26.869304 30719 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:26.869858 30719 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:26.874094 30719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99: Generated new TSK 0
I20260807 22:35:26.874279 30719 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:26.907208 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43905
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:27.001681 30727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:27.001883 30727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:27.001907 30727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:27.003125 30727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:27.003170 30727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:27.003190 30727 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:27.003208 30727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:27.005156 30727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:27.005399 30727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:27.005641 30727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:27.011842 30740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:27.011919 30734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:27.011883 30735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:27.012521 30727 server_base.cc:1034] running on GCE node
I20260807 22:35:27.012936 30727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:27.013232 30727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:27.014408 30727 hybrid_clock.cc:648] HybridClock initialized: now 1786142127014387 us; error 41 us; skew 500 ppm
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:27.017215 30727 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:27.017725 30727 webserver.cc:460] Webserver started at http://127.28.82.193:42723/ using document root <none> and password file <none>
I20260807 22:35:27.017994 30727 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:27.018045 30727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:27.018220 30727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:27.019414 30727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "369e4f01677b48638d033b07b9ed0a80"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "d48f8a56f33ad5f6bdfb75de294ce594"
server_key_iv: "744ae3e6cccbef3d564d0ce61ed50d53"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.019817 30727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "369e4f01677b48638d033b07b9ed0a80"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "d48f8a56f33ad5f6bdfb75de294ce594"
server_key_iv: "744ae3e6cccbef3d564d0ce61ed50d53"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.021265 30727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:35:27.022099 30752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.022295 30727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:27.022377 30727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "369e4f01677b48638d033b07b9ed0a80"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "d48f8a56f33ad5f6bdfb75de294ce594"
server_key_iv: "744ae3e6cccbef3d564d0ce61ed50d53"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.022432 30727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:27.033727 30727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:27.036031 30727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:27.036154 30727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:27.036625 30727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:27.036664 30727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.036686 30727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:27.036741 30727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:27.043920 30727 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:44307
I20260807 22:35:27.044094 30915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:44307 every 8 connection(s)
I20260807 22:35:27.044348 30727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:27.046680 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.037240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:38287 (local address 127.28.82.254:43905)
0807 22:35:27.037312 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.037314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.037485 (+ 171us) server_negotiation.cc:407] Connection header received
0807 22:35:27.037997 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.037999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.038015 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.038114 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.039017 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.039871 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.040897 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.041245 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.041322 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.043938 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.043939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.043940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.043948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.045469 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.045833 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.045834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.045835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.045889 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.046108 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.046108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.046109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.046208 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.046259 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.046511 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.046611 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:35:27.046804 30766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.037158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43905 (local address 127.28.82.193:38287)
0807 22:35:27.037361 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.037401 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.037901 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.038167 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.038170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.038586 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:27.038904 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.038911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.040001 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.040002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.040803 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.040805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.041340 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.041342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.041486 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.042085 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:27.042094 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:27.043841 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:27.045540 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.045547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.045548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.045770 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.045945 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.045945 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.045945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.046044 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.046260 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:27.046264 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:27.046374 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.046451 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.046627 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260807 22:35:27.047305 30916 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43905
I20260807 22:35:27.047407 30916 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:27.047575 30916 heartbeater.cc:511] Master 127.28.82.254:43905 requested a full tablet report, sending...
I20260807 22:35:27.048146 30628 ts_manager.cc:194] Registered new tserver with Master: 369e4f01677b48638d033b07b9ed0a80 (127.28.82.193:44307)
I20260807 22:35:27.049077 30628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:38287
I20260807 22:35:27.053309 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 30727
I20260807 22:35:27.053422 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:27.053630 29003 external_mini_cluster.cc:1428] Setting key fea5a07cd910ffdc97d15ff40366cfbe
I20260807 22:35:27.054286 30922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.048406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43905 (local address 127.28.82.193:50247)
0807 22:35:27.048538 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.048549 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.048617 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.048753 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.048754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.048948 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:27.049005 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.049007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.049913 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.049914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.050393 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.050395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.050784 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.050785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.050843 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.051306 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:27.051311 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:27.051584 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:27.053180 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.053181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.053182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.053419 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.053669 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.053670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.053670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.053736 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.054037 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:27.054037 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:27.054077 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.054135 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.054223 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260807 22:35:27.054335 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.048492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:50247 (local address 127.28.82.254:43905)
0807 22:35:27.048532 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.048534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.048603 (+ 69us) server_negotiation.cc:407] Connection header received
0807 22:35:27.048669 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.048669 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.048677 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.048747 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.049074 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.049796 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.050494 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.050700 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.050775 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.051658 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.051659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.051659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.051665 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.052959 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.053525 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.053526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.053527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.053589 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.053832 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.053833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.053834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.053971 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.054038 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.054173 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.054249 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:27.079270 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43905
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:27.161618 30925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:27.161851 30925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:27.161890 30925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:27.163093 30925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:27.163156 30925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:27.163190 30925 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:27.163228 30925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:27.164664 30925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:27.164940 30925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:27.165170 30925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:27.170343 30930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:27.170631 30925 server_base.cc:1034] running on GCE node
W20260807 22:35:27.170328 30935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:27.170356 30933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:27.170964 30925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:27.171195 30925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:27.172353 30925 hybrid_clock.cc:648] HybridClock initialized: now 1786142127172339 us; error 28 us; skew 500 ppm
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:27.175041 30925 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:27.175499 30925 webserver.cc:460] Webserver started at http://127.28.82.194:36685/ using document root <none> and password file <none>
I20260807 22:35:27.175700 30925 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:27.175745 30925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:27.175832 30925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:27.176844 30925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "5c2cdc81c0994ac79a2615846df2ad42"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "4c5a70de268e88a92abc72390e615b44"
server_key_iv: "d9f23772626048da5d851409cd59032e"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.177151 30925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "5c2cdc81c0994ac79a2615846df2ad42"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "4c5a70de268e88a92abc72390e615b44"
server_key_iv: "d9f23772626048da5d851409cd59032e"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.178429 30925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 22:35:27.179222 30944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.179394 30925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:27.179469 30925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "5c2cdc81c0994ac79a2615846df2ad42"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "4c5a70de268e88a92abc72390e615b44"
server_key_iv: "d9f23772626048da5d851409cd59032e"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.179522 30925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:27.205999 30925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:27.208307 30925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:27.208480 30925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:27.208994 30925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:27.209053 30925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.209089 30925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:27.209144 30925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.216105 30925 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:33857
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:27.216197 31117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:33857 every 8 connection(s)
I20260807 22:35:27.216480 30925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:27.219287 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.209475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:58553 (local address 127.28.82.254:43905)
0807 22:35:27.209549 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.209552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.209742 (+ 190us) server_negotiation.cc:407] Connection header received
0807 22:35:27.210226 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.210228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.210241 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.210324 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.211105 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.211973 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.213556 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.213920 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.214012 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.216715 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.216717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.216717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.216725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.217894 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.218287 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.218288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.218288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.218348 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.218646 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.218647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.218648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.218780 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.218856 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.219117 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.219197 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:35:27.219462 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.209436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43905 (local address 127.28.82.194:58553)
0807 22:35:27.209593 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.209632 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.210094 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.210381 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.210384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.210707 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:27.210995 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.211002 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.212084 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.212085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.213442 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.213445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.214028 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.214029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.214215 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.214771 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:27.214779 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:27.216615 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:27.217978 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.217984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.217987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.218210 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.218421 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.218422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.218423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.218572 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.218846 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:27.218853 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:27.218984 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.219080 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.219238 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260807 22:35:27.219934 31119 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43905
I20260807 22:35:27.220018 31119 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:27.220202 31119 heartbeater.cc:511] Master 127.28.82.254:43905 requested a full tablet report, sending...
I20260807 22:35:27.220564 30629 ts_manager.cc:194] Registered new tserver with Master: 5c2cdc81c0994ac79a2615846df2ad42 (127.28.82.194:33857)
I20260807 22:35:27.220944 30629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:58553
I20260807 22:35:27.224200 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 30925
I20260807 22:35:27.224296 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:27.224488 29003 external_mini_cluster.cc:1428] Setting key 66705af40ca4a28300965813244b716e
I20260807 22:35:27.226672 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.221079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:48435 (local address 127.28.82.254:43905)
0807 22:35:27.221117 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.221118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.221352 (+ 234us) server_negotiation.cc:407] Connection header received
0807 22:35:27.221412 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.221413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.221422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.221469 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.221813 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.222274 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.222995 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.223164 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.223211 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.223929 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.223930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.223930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.223936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.225434 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.225821 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.225821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.225822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.225872 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.226119 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.226120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.226120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.226292 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.226343 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.226550 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.226617 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:35:27.226774 31125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.220988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43905 (local address 127.28.82.194:48435)
0807 22:35:27.221278 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.221287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.221341 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.221483 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.221484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.221670 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:27.221744 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.221745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.222357 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.222358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.222924 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.222926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.223222 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.223222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.223274 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.223589 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:27.223593 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:27.223858 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:27.225507 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.225508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.225508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.225746 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.225965 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.225966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.225966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.226041 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.226353 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:27.226354 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:27.226401 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.226495 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.226603 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:27.251396 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43905
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:27.336225 31130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:27.336465 31130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:27.336506 31130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:27.337741 31130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:27.337805 31130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:27.337838 31130 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:27.337877 31130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:27.339291 31130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:43905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:27.339547 31130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:27.339775 31130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:27.344841 31137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:27.345053 31130 server_base.cc:1034] running on GCE node
W20260807 22:35:27.344822 31138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:27.344822 31142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:27.345317 31130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:27.345558 31130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:27.346719 31130 hybrid_clock.cc:648] HybridClock initialized: now 1786142127346702 us; error 26 us; skew 500 ppm
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:27.349314 31130 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:27.349783 31130 webserver.cc:460] Webserver started at http://127.28.82.195:46323/ using document root <none> and password file <none>
I20260807 22:35:27.350000 31130 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:27.350066 31130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:27.350220 31130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:27.351353 31130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "aa29b3e1be0d411dbb6a9e273c83ade6"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "9abb116a5539d0dd4b4eab37b0f0269c"
server_key_iv: "65c3f089eb98108b4d363075eec6a930"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.351778 31130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "aa29b3e1be0d411dbb6a9e273c83ade6"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "9abb116a5539d0dd4b4eab37b0f0269c"
server_key_iv: "65c3f089eb98108b4d363075eec6a930"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.353158 31130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:35:27.353863 31152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.354027 31130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:27.354107 31130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "aa29b3e1be0d411dbb6a9e273c83ade6"
format_stamp: "Formatted at 2026-08-07 22:35:27 on dist-test-slave-7kzw"
server_key: "9abb116a5539d0dd4b4eab37b0f0269c"
server_key_iv: "65c3f089eb98108b4d363075eec6a930"
server_key_version: "encryptionkey@0"
I20260807 22:35:27.354171 31130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:27.366868 31130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:27.369026 31130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:27.369134 31130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:27.369566 31130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:27.369606 31130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.369628 31130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:27.369666 31130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:27.376869 31130 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:45875
I20260807 22:35:27.376936 31327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:45875 every 8 connection(s)
I20260807 22:35:27.377251 31130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:27.380282 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.370042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:48309 (local address 127.28.82.254:43905)
0807 22:35:27.370112 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.370114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.370270 (+ 156us) server_negotiation.cc:407] Connection header received
0807 22:35:27.370808 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.370809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.370823 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.370908 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.371695 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.372552 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.374089 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.374439 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.374557 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.377285 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.377287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.377287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.377297 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.378889 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.379317 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.379319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.379319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.379385 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.379631 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.379632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.379632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.379763 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.379834 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.380100 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.380181 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:35:27.380455 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.369994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43905 (local address 127.28.82.195:48309)
0807 22:35:27.370149 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.370187 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.370713 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.370956 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.370959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.371308 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:27.371587 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.371594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.372657 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.372658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.373976 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.373980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.374667 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.374669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.374833 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.375302 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:27.375311 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:27.377191 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:27.378990 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.378997 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.378999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.379224 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.379456 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.379456 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.379457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.379567 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.379838 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:27.379845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:27.379975 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.380077 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.380232 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260807 22:35:27.380898 31328 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43905
I20260807 22:35:27.380983 31328 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:27.381170 31328 heartbeater.cc:511] Master 127.28.82.254:43905 requested a full tablet report, sending...
I20260807 22:35:27.381536 30628 ts_manager.cc:194] Registered new tserver with Master: aa29b3e1be0d411dbb6a9e273c83ade6 (127.28.82.195:45875)
I20260807 22:35:27.382014 30628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:48309
I20260807 22:35:27.386432 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 31130
I20260807 22:35:27.386561 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:27.386754 29003 external_mini_cluster.cc:1428] Setting key b0913b407f13faf76164811d9ada0cb6
I20260807 22:35:27.387247 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.382016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:59303 (local address 127.28.82.254:43905)
0807 22:35:27.382080 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.382082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.382166 (+ 84us) server_negotiation.cc:407] Connection header received
0807 22:35:27.382228 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.382229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.382239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.382303 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.382690 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.383152 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.383829 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.384031 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.384102 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.384895 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.384897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.384897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.384904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.386091 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.386437 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.386438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.386438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.386518 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.386728 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.386730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.386730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.386869 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.386929 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.387096 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.387164 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:35:27.387277 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.381929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43905 (local address 127.28.82.195:59303)
0807 22:35:27.382103 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.382114 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.382180 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.382310 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.382310 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.382551 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:27.382608 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.382610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.383249 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.383250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.383738 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.383740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.384101 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.384102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.384177 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.384529 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:27.384533 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:27.384803 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:27.386175 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.386176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.386177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.386352 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.386594 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.386595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.386595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.386656 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.386934 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:27.386935 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:27.386979 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.387054 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.387156 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260807 22:35:27.387611 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:27.411232 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.404827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.28.82.254:43905)
0807 22:35:27.404885 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.404887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.404895 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:27.404915 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.404916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.404924 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.404974 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.405494 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.405956 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.406715 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.406918 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.406982 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.409065 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.409066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.409066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.409073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.410278 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.410645 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.410646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.410646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.410699 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.410854 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.410855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.410855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.410941 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.411010 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.411094 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.411151 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:35:27.412809 30628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:27.414623 30628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:27.421429 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.417030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:33857 (local address 127.0.0.1:38214)
0807 22:35:27.417246 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.417291 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.417356 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.417732 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.417733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.417735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.417814 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.417816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.419158 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.419159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.420296 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.420299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.421087 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.421088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.421181 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.421194 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.421292 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.421345 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":85,"thread_start_us":65,"threads_started":1}
I20260807 22:35:27.421446 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.417169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38214 (local address 127.28.82.194:33857)
0807 22:35:27.417568 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.417573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.417584 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:35:27.417623 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.417625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.417641 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.417718 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.417907 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.419048 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.420379 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.420994 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.421078 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.421143 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.421309 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.421352 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":87,"threads_started":1}
I20260807 22:35:27.421591 31353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.417292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:45875 (local address 127.0.0.1:54322)
0807 22:35:27.417493 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.417503 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.417539 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.418057 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.418058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.418060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.418118 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.418119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.419094 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.419095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.420433 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.420435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.421357 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.421358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.421428 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.421432 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.421462 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.421509 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":79,"thread_start_us":80,"threads_started":1}
I20260807 22:35:27.421594 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.417089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:44307 (local address 127.0.0.1:51504)
0807 22:35:27.417351 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.417362 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.417414 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.417665 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.417669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.417671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.417741 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.417743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.419161 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.419162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.420468 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.420469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.421421 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.421422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.421477 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.421480 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.421507 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.421536 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":122,"thread_start_us":105,"threads_started":1}
I20260807 22:35:27.421999 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.417508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.28.82.195:45875)
0807 22:35:27.417878 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.417882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.417892 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:27.417941 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.417944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.417959 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.418191 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.418266 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.419012 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.420517 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.421247 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.421754 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.421834 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.421863 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.421903 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":93,"threads_started":1}
I20260807 22:35:27.422156 31014 tablet_service.cc:1467] Processing CreateTablet for tablet fbe706c1924147a98f3f65818fe9cf3e (DEFAULT_TABLE table=test-table [id=fb5a8abd9c9f4ce39d256a8886784450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:27.422394 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.417218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.28.82.193:44307)
0807 22:35:27.417427 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.417432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.417442 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:27.417481 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.417484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.417500 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.417628 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.417852 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.419047 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.420567 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.421339 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.421889 (+ 550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.421943 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.421969 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.422006 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
I20260807 22:35:27.422559 31014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbe706c1924147a98f3f65818fe9cf3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.422626 31226 tablet_service.cc:1467] Processing CreateTablet for tablet fbe706c1924147a98f3f65818fe9cf3e (DEFAULT_TABLE table=test-table [id=fb5a8abd9c9f4ce39d256a8886784450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:27.422952 31226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbe706c1924147a98f3f65818fe9cf3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.422993 30826 tablet_service.cc:1467] Processing CreateTablet for tablet fbe706c1924147a98f3f65818fe9cf3e (DEFAULT_TABLE table=test-table [id=fb5a8abd9c9f4ce39d256a8886784450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:27.423249 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbe706c1924147a98f3f65818fe9cf3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.426335 31358 tablet_bootstrap.cc:492] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Bootstrap starting.
I20260807 22:35:27.426579 31357 tablet_bootstrap.cc:492] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Bootstrap starting.
I20260807 22:35:27.426978 31358 tablet_bootstrap.cc:654] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.427294 31358 log.cc:824] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:27.427534 31357 tablet_bootstrap.cc:654] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.427886 31357 log.cc:824] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:27.428081 31358 tablet_bootstrap.cc:492] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: No bootstrap required, opened a new log
I20260807 22:35:27.428108 31359 tablet_bootstrap.cc:492] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Bootstrap starting.
I20260807 22:35:27.428159 31358 ts_tablet_manager.cc:1397] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:27.428774 31359 tablet_bootstrap.cc:654] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.429044 31359 log.cc:824] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:27.429579 31357 tablet_bootstrap.cc:492] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: No bootstrap required, opened a new log
I20260807 22:35:27.429641 31357 ts_tablet_manager.cc:1397] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:35:27.429999 31358 raft_consensus.cc:348] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.430150 31358 raft_consensus.cc:374] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.430200 31358 raft_consensus.cc:729] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369e4f01677b48638d033b07b9ed0a80, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.430310 31358 consensus_queue.cc:260] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.430624 31358 ts_tablet_manager.cc:1428] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260807 22:35:27.430769 30916 heartbeater.cc:503] Master 127.28.82.254:43905 was elected leader, sending a full tablet report...
I20260807 22:35:27.430840 31359 tablet_bootstrap.cc:492] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: No bootstrap required, opened a new log
I20260807 22:35:27.430908 31359 ts_tablet_manager.cc:1397] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 22:35:27.431879 31357 raft_consensus.cc:348] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.432039 31357 raft_consensus.cc:374] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.432073 31357 raft_consensus.cc:729] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa29b3e1be0d411dbb6a9e273c83ade6, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.432188 31357 consensus_queue.cc:260] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.432452 31357 ts_tablet_manager.cc:1428] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:27.432492 31328 heartbeater.cc:503] Master 127.28.82.254:43905 was elected leader, sending a full tablet report...
I20260807 22:35:27.432941 31359 raft_consensus.cc:348] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.433077 31359 raft_consensus.cc:374] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.433131 31359 raft_consensus.cc:729] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2cdc81c0994ac79a2615846df2ad42, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.433241 31359 consensus_queue.cc:260] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.433573 31359 ts_tablet_manager.cc:1428] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:27.433614 31119 heartbeater.cc:503] Master 127.28.82.254:43905 was elected leader, sending a full tablet report...
W20260807 22:35:27.467120 31120 tablet.cc:2369] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:35:27.545135 30917 tablet.cc:2369] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:27.608944 31363 raft_consensus.cc:482] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:27.609066 31363 raft_consensus.cc:504] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.609508 31363 leader_election.cc:288] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c2cdc81c0994ac79a2615846df2ad42 (127.28.82.194:33857), aa29b3e1be0d411dbb6a9e273c83ade6 (127.28.82.195:45875)
I20260807 22:35:27.612651 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.609825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:60929 (local address 127.28.82.194:33857)
0807 22:35:27.609874 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.609876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.609954 (+ 78us) server_negotiation.cc:407] Connection header received
0807 22:35:27.610017 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.610018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.610028 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.610089 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.610322 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.610812 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.611852 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.612190 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.612392 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.612402 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.612524 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.612578 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:35:27.612663 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.609840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:53689 (local address 127.28.82.195:45875)
0807 22:35:27.609884 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.609886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.609955 (+ 69us) server_negotiation.cc:407] Connection header received
0807 22:35:27.610032 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.610033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.610047 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.610116 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.610276 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.610816 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.611796 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.612179 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.612238 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.612245 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.612562 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.612600 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:35:27.612679 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.609747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:33857 (local address 127.28.82.193:60929)
0807 22:35:27.609879 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.609890 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.609956 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.610114 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.610115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.610117 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.610229 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.610244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.610898 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.610899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.611732 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.611735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.612392 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.612393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.612455 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.612459 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.612503 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.612572 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":71,"thread_start_us":66,"threads_started":1}
I20260807 22:35:27.612679 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.609731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:45875 (local address 127.28.82.193:53689)
0807 22:35:27.609874 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.609886 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.609954 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.610138 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.610139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.610141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.610203 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.610205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.610900 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.610902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.611701 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.611703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.612385 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.612386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.612462 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.612466 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.612541 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.612585 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260807 22:35:27.613252 31255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbe706c1924147a98f3f65818fe9cf3e" candidate_uuid: "369e4f01677b48638d033b07b9ed0a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" is_pre_election: true
I20260807 22:35:27.613248 31044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbe706c1924147a98f3f65818fe9cf3e" candidate_uuid: "369e4f01677b48638d033b07b9ed0a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c2cdc81c0994ac79a2615846df2ad42" is_pre_election: true
I20260807 22:35:27.613408 31044 raft_consensus.cc:2454] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 369e4f01677b48638d033b07b9ed0a80 in term 0.
I20260807 22:35:27.613408 31255 raft_consensus.cc:2454] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 369e4f01677b48638d033b07b9ed0a80 in term 0.
I20260807 22:35:27.613641 30753 leader_election.cc:302] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 369e4f01677b48638d033b07b9ed0a80, 5c2cdc81c0994ac79a2615846df2ad42; no voters:
I20260807 22:35:27.613799 31363 raft_consensus.cc:2781] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:27.613876 31363 raft_consensus.cc:482] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:27.613936 31363 raft_consensus.cc:3037] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.614770 31363 raft_consensus.cc:504] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.614918 31363 leader_election.cc:288] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [CANDIDATE]: Term 1 election: Requested vote from peers 5c2cdc81c0994ac79a2615846df2ad42 (127.28.82.194:33857), aa29b3e1be0d411dbb6a9e273c83ade6 (127.28.82.195:45875)
I20260807 22:35:27.615082 31044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbe706c1924147a98f3f65818fe9cf3e" candidate_uuid: "369e4f01677b48638d033b07b9ed0a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c2cdc81c0994ac79a2615846df2ad42"
I20260807 22:35:27.615106 31255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbe706c1924147a98f3f65818fe9cf3e" candidate_uuid: "369e4f01677b48638d033b07b9ed0a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6"
I20260807 22:35:27.615181 31255 raft_consensus.cc:3037] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.615170 31044 raft_consensus.cc:3037] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.616066 31044 raft_consensus.cc:2454] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 369e4f01677b48638d033b07b9ed0a80 in term 1.
I20260807 22:35:27.616066 31255 raft_consensus.cc:2454] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 369e4f01677b48638d033b07b9ed0a80 in term 1.
I20260807 22:35:27.616254 30757 leader_election.cc:302] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 369e4f01677b48638d033b07b9ed0a80, aa29b3e1be0d411dbb6a9e273c83ade6; no voters:
I20260807 22:35:27.616358 31363 raft_consensus.cc:2781] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:27.616463 31363 raft_consensus.cc:686] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 1 LEADER]: Becoming Leader. State: Replica: 369e4f01677b48638d033b07b9ed0a80, State: Running, Role: LEADER
I20260807 22:35:27.616582 31363 consensus_queue.cc:237] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.617379 30629 catalog_manager.cc:5495] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 369e4f01677b48638d033b07b9ed0a80 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "369e4f01677b48638d033b07b9ed0a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } health_report { overall_health: UNKNOWN } } }
W20260807 22:35:27.628100 31330 tablet.cc:2369] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:27.648387 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.645584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.28.82.193:44307)
0807 22:35:27.645682 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.645684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.645756 (+ 72us) server_negotiation.cc:407] Connection header received
0807 22:35:27.645876 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.645877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.645898 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.645974 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:27.646142 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.646637 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.647513 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.647738 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.647817 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.648021 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:27.648124 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:27.648212 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.648259 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.648281 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 22:35:27.650291 31255 raft_consensus.cc:1261] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Refusing update from remote peer 369e4f01677b48638d033b07b9ed0a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:27.650312 31044 raft_consensus.cc:1261] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Refusing update from remote peer 369e4f01677b48638d033b07b9ed0a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:27.650684 31363 consensus_queue.cc:1035] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:27.650853 31369 consensus_queue.cc:1035] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:27.654172 31373 mvcc.cc:205] Tried to move back new op lower bound from 7316038154852458496 to 7316038154718179328. Current Snapshot: MvccSnapshot[applied={T|T < 7316038154852458496}]
I20260807 22:35:27.657685 30629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:27.657847 30629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:27.658811 30629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99: Sending DeleteTablet for 3 replicas of tablet fbe706c1924147a98f3f65818fe9cf3e
I20260807 22:35:27.659168 30826 tablet_service.cc:1514] Processing DeleteTablet for tablet fbe706c1924147a98f3f65818fe9cf3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:27 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:51504
I20260807 22:35:27.659220 31226 tablet_service.cc:1514] Processing DeleteTablet for tablet fbe706c1924147a98f3f65818fe9cf3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:27 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:54322
I20260807 22:35:27.659408 31014 tablet_service.cc:1514] Processing DeleteTablet for tablet fbe706c1924147a98f3f65818fe9cf3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:27 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:38214
I20260807 22:35:27.659543 30629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 22:35:27.659631 31391 tablet_replica.cc:330] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: stopping tablet replica
W20260807 22:35:27.659700 30629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:27.659731 31391 raft_consensus.cc:2229] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:27.659751 31388 tablet_replica.cc:330] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: stopping tablet replica
I20260807 22:35:27.659830 31388 raft_consensus.cc:2229] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:27.659842 31391 raft_consensus.cc:2258] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:27.660225 31389 tablet_replica.cc:330] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: stopping tablet replica
I20260807 22:35:27.660317 31391 ts_tablet_manager.cc:1905] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:27.660416 31389 raft_consensus.cc:2229] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:27.660579 31389 raft_consensus.cc:2258] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:27.661075 31389 ts_tablet_manager.cc:1905] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:27.662734 31391 ts_tablet_manager.cc:1918] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:27.662820 31389 ts_tablet_manager.cc:1918] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:27.662873 31391 log.cc:1196] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/fbe706c1924147a98f3f65818fe9cf3e
I20260807 22:35:27.662941 31389 log.cc:1196] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/fbe706c1924147a98f3f65818fe9cf3e
I20260807 22:35:27.663260 31389 ts_tablet_manager.cc:1939] T fbe706c1924147a98f3f65818fe9cf3e P aa29b3e1be0d411dbb6a9e273c83ade6: Deleting consensus metadata
I20260807 22:35:27.663297 31391 ts_tablet_manager.cc:1939] T fbe706c1924147a98f3f65818fe9cf3e P 5c2cdc81c0994ac79a2615846df2ad42: Deleting consensus metadata
I20260807 22:35:27.663736 30608 catalog_manager.cc:4841] TS aa29b3e1be0d411dbb6a9e273c83ade6 (127.28.82.195:45875): tablet fbe706c1924147a98f3f65818fe9cf3e (table test-table [id=fb5a8abd9c9f4ce39d256a8886784450]) successfully deleted
I20260807 22:35:27.664265 30605 catalog_manager.cc:4841] TS 5c2cdc81c0994ac79a2615846df2ad42 (127.28.82.194:33857): tablet fbe706c1924147a98f3f65818fe9cf3e (table test-table [id=fb5a8abd9c9f4ce39d256a8886784450]) successfully deleted
I20260807 22:35:27.665731 31388 raft_consensus.cc:2258] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:27.666559 30826 tablet_service.cc:1467] Processing CreateTablet for tablet ab4ee4a206ff4963915cb0840ce4a037 (DEFAULT_TABLE table=test-table [id=bdbddb3c883f441abd9356cf87ec44da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:27.666636 31226 tablet_service.cc:1467] Processing CreateTablet for tablet ab4ee4a206ff4963915cb0840ce4a037 (DEFAULT_TABLE table=test-table [id=bdbddb3c883f441abd9356cf87ec44da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:27.666688 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4ee4a206ff4963915cb0840ce4a037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.666725 31226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4ee4a206ff4963915cb0840ce4a037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.668179 31357 tablet_bootstrap.cc:492] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: Bootstrap starting.
I20260807 22:35:27.668274 31014 tablet_service.cc:1467] Processing CreateTablet for tablet ab4ee4a206ff4963915cb0840ce4a037 (DEFAULT_TABLE table=test-table [id=bdbddb3c883f441abd9356cf87ec44da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:27.668378 31014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4ee4a206ff4963915cb0840ce4a037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.668601 31357 tablet_bootstrap.cc:654] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.669688 31388 ts_tablet_manager.cc:1905] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:27.669744 31359 tablet_bootstrap.cc:492] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: Bootstrap starting.
I20260807 22:35:27.670156 31359 tablet_bootstrap.cc:654] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.670939 31359 tablet_bootstrap.cc:492] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: No bootstrap required, opened a new log
I20260807 22:35:27.670986 31359 ts_tablet_manager.cc:1397] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:27.671150 31359 raft_consensus.cc:348] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.671231 31359 raft_consensus.cc:374] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.671257 31359 raft_consensus.cc:729] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2cdc81c0994ac79a2615846df2ad42, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.671300 31359 consensus_queue.cc:260] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.671387 31359 ts_tablet_manager.cc:1428] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.671773 31388 ts_tablet_manager.cc:1918] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:27.671873 31388 log.cc:1196] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/fbe706c1924147a98f3f65818fe9cf3e
I20260807 22:35:27.672186 31388 ts_tablet_manager.cc:1939] T fbe706c1924147a98f3f65818fe9cf3e P 369e4f01677b48638d033b07b9ed0a80: Deleting consensus metadata
I20260807 22:35:27.672624 30608 catalog_manager.cc:4841] TS 369e4f01677b48638d033b07b9ed0a80 (127.28.82.193:44307): tablet fbe706c1924147a98f3f65818fe9cf3e (table test-table [id=fb5a8abd9c9f4ce39d256a8886784450]) successfully deleted
I20260807 22:35:27.672962 31357 tablet_bootstrap.cc:492] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: No bootstrap required, opened a new log
I20260807 22:35:27.673009 31357 ts_tablet_manager.cc:1397] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 22:35:27.673163 31357 raft_consensus.cc:348] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.673229 31357 raft_consensus.cc:374] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.673249 31357 raft_consensus.cc:729] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa29b3e1be0d411dbb6a9e273c83ade6, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.673288 31357 consensus_queue.cc:260] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.673398 31357 ts_tablet_manager.cc:1428] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:27.678529 31358 tablet_bootstrap.cc:492] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: Bootstrap starting.
I20260807 22:35:27.678952 31358 tablet_bootstrap.cc:654] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.679662 31358 tablet_bootstrap.cc:492] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: No bootstrap required, opened a new log
I20260807 22:35:27.679719 31358 ts_tablet_manager.cc:1397] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:35:27.679852 31358 raft_consensus.cc:348] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.679922 31358 raft_consensus.cc:374] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.679943 31358 raft_consensus.cc:729] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369e4f01677b48638d033b07b9ed0a80, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.680001 31358 consensus_queue.cc:260] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.680088 31358 ts_tablet_manager.cc:1428] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.719822 31365 raft_consensus.cc:482] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:27.719944 31365 raft_consensus.cc:504] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.720321 31365 leader_election.cc:288] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 369e4f01677b48638d033b07b9ed0a80 (127.28.82.193:44307), 5c2cdc81c0994ac79a2615846df2ad42 (127.28.82.194:33857)
I20260807 22:35:27.723881 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.720577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:38471 (local address 127.28.82.194:33857)
0807 22:35:27.720649 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.720651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.720724 (+ 73us) server_negotiation.cc:407] Connection header received
0807 22:35:27.720853 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.720854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.720864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.720931 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.721167 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.721868 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.723144 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.723503 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.723585 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.723597 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.723763 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.723802 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:35:27.723939 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.720474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:44307 (local address 127.28.82.195:60967)
0807 22:35:27.720574 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.720586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.720644 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.720926 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.720927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.720929 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.721003 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.721005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.721684 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.721685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.722872 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.722875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.723647 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.723648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.723736 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.723740 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.723794 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.723839 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 22:35:27.723950 31395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.720475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:33857 (local address 127.28.82.195:38471)
0807 22:35:27.720647 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.720659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.720723 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.720936 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.720937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.720939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.721026 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.721028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.721968 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.721969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.723062 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.723064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.723604 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.723606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.723686 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.723689 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.723754 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.723796 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":15,"thread_start_us":92,"threads_started":1}
I20260807 22:35:27.724222 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.720722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:60967 (local address 127.28.82.193:44307)
0807 22:35:27.720779 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.720781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.720788 (+ 7us) server_negotiation.cc:407] Connection header received
0807 22:35:27.720814 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.720815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.720826 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.720876 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.721082 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.721572 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.722975 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.723511 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.723588 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.723597 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.724115 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.724156 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 22:35:27.724300 31044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4ee4a206ff4963915cb0840ce4a037" candidate_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c2cdc81c0994ac79a2615846df2ad42" is_pre_election: true
I20260807 22:35:27.724383 31044 raft_consensus.cc:2454] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa29b3e1be0d411dbb6a9e273c83ade6 in term 0.
I20260807 22:35:27.724511 30851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4ee4a206ff4963915cb0840ce4a037" candidate_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "369e4f01677b48638d033b07b9ed0a80" is_pre_election: true
I20260807 22:35:27.724642 30851 raft_consensus.cc:2454] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa29b3e1be0d411dbb6a9e273c83ade6 in term 0.
I20260807 22:35:27.724660 31153 leader_election.cc:302] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c2cdc81c0994ac79a2615846df2ad42, aa29b3e1be0d411dbb6a9e273c83ade6; no voters:
I20260807 22:35:27.724803 31365 raft_consensus.cc:2781] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:27.724848 31365 raft_consensus.cc:482] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:27.724871 31365 raft_consensus.cc:3037] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.725616 31365 raft_consensus.cc:504] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.725777 31365 leader_election.cc:288] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [CANDIDATE]: Term 1 election: Requested vote from peers 369e4f01677b48638d033b07b9ed0a80 (127.28.82.193:44307), 5c2cdc81c0994ac79a2615846df2ad42 (127.28.82.194:33857)
I20260807 22:35:27.725875 30851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4ee4a206ff4963915cb0840ce4a037" candidate_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "369e4f01677b48638d033b07b9ed0a80"
I20260807 22:35:27.725946 30851 raft_consensus.cc:3037] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.725975 31044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4ee4a206ff4963915cb0840ce4a037" candidate_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c2cdc81c0994ac79a2615846df2ad42"
I20260807 22:35:27.726044 31044 raft_consensus.cc:3037] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.726729 30851 raft_consensus.cc:2454] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa29b3e1be0d411dbb6a9e273c83ade6 in term 1.
I20260807 22:35:27.726750 31044 raft_consensus.cc:2454] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa29b3e1be0d411dbb6a9e273c83ade6 in term 1.
I20260807 22:35:27.726917 31158 leader_election.cc:302] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 369e4f01677b48638d033b07b9ed0a80, aa29b3e1be0d411dbb6a9e273c83ade6; no voters:
I20260807 22:35:27.727016 31365 raft_consensus.cc:2781] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:27.727147 31365 raft_consensus.cc:686] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 LEADER]: Becoming Leader. State: Replica: aa29b3e1be0d411dbb6a9e273c83ade6, State: Running, Role: LEADER
I20260807 22:35:27.727213 31365 consensus_queue.cc:237] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.727963 30629 catalog_manager.cc:5495] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa29b3e1be0d411dbb6a9e273c83ade6 (127.28.82.195). New cstate: current_term: 1 leader_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } health_report { overall_health: HEALTHY } } }
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142126, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:27.771183 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.763278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43905 (local address 127.0.0.1:40488)
0807 22:35:27.763484 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.763497 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.763578 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.763821 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.763823 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.764299 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:27.764365 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.764368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.765375 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.765376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.765895 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.765897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.766382 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.766383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.766428 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.767036 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:27.767041 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:27.768548 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:27.770168 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.770175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.770177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.770366 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.770592 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:27.770593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:27.770593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:27.770660 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:27.770917 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:27.770924 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:27.770964 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.771013 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.771101 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
I20260807 22:35:27.771322 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.763418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40488 (local address 127.28.82.254:43905)
0807 22:35:27.763502 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.763504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.763595 (+ 91us) server_negotiation.cc:407] Connection header received
0807 22:35:27.763680 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.763681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.763694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.763805 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.764494 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.765273 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.765979 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.766268 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.766345 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.768632 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.768634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.768634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.768642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.770086 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.770436 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.770437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.770438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.770523 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.770717 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.770718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.770719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.770849 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.770912 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.771056 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.771173 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":37}
I20260807 22:35:27.773531 30629 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40488:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:35:27.773762 30629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:27.775275 30826 tablet_service.cc:1467] Processing CreateTablet for tablet 829160dfae0b427c89cc6b759a59103d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd14131403e9433bb82d9f90d600dee1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:27.775352 31014 tablet_service.cc:1467] Processing CreateTablet for tablet 829160dfae0b427c89cc6b759a59103d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd14131403e9433bb82d9f90d600dee1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:27.775379 31226 tablet_service.cc:1467] Processing CreateTablet for tablet 829160dfae0b427c89cc6b759a59103d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd14131403e9433bb82d9f90d600dee1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:27.775447 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 829160dfae0b427c89cc6b759a59103d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.775475 31014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 829160dfae0b427c89cc6b759a59103d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.775542 31226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 829160dfae0b427c89cc6b759a59103d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:27.777606 31358 tablet_bootstrap.cc:492] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: Bootstrap starting.
I20260807 22:35:27.777971 31358 tablet_bootstrap.cc:654] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.779300 31359 tablet_bootstrap.cc:492] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: Bootstrap starting.
I20260807 22:35:27.779904 31357 tablet_bootstrap.cc:492] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: Bootstrap starting.
I20260807 22:35:27.780014 31359 tablet_bootstrap.cc:654] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.780217 31358 tablet_bootstrap.cc:492] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: No bootstrap required, opened a new log
I20260807 22:35:27.780263 31358 ts_tablet_manager.cc:1397] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260807 22:35:27.780372 31357 tablet_bootstrap.cc:654] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:27.780369 31358 raft_consensus.cc:348] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.780434 31358 raft_consensus.cc:374] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.780453 31358 raft_consensus.cc:729] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369e4f01677b48638d033b07b9ed0a80, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.780495 31358 consensus_queue.cc:260] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.780571 31358 ts_tablet_manager.cc:1428] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:27.780941 31359 tablet_bootstrap.cc:492] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: No bootstrap required, opened a new log
I20260807 22:35:27.780877 31384 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.780979 31359 ts_tablet_manager.cc:1397] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:27.780974 31384 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:27.781138 31359 raft_consensus.cc:348] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.781219 31359 raft_consensus.cc:374] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.781237 31359 raft_consensus.cc:729] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2cdc81c0994ac79a2615846df2ad42, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.781275 31359 consensus_queue.cc:260] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.781352 31359 ts_tablet_manager.cc:1428] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:27.781529 31357 tablet_bootstrap.cc:492] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: No bootstrap required, opened a new log
I20260807 22:35:27.781481 31366 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.781563 31366 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:27.781575 31357 ts_tablet_manager.cc:1397] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:27.781901 31357 raft_consensus.cc:348] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.781983 31357 raft_consensus.cc:374] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:27.782006 31357 raft_consensus.cc:729] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa29b3e1be0d411dbb6a9e273c83ade6, State: Initialized, Role: FOLLOWER
I20260807 22:35:27.782048 31357 consensus_queue.cc:260] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.782150 31357 ts_tablet_manager.cc:1428] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:27.782195 31365 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.782280 31365 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:27.812439 31366 raft_consensus.cc:482] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:27.812570 31366 raft_consensus.cc:504] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.812947 31366 leader_election.cc:288] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 369e4f01677b48638d033b07b9ed0a80 (127.28.82.193:44307), aa29b3e1be0d411dbb6a9e273c83ade6 (127.28.82.195:45875)
I20260807 22:35:27.816274 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.813256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:52853 (local address 127.28.82.193:44307)
0807 22:35:27.813328 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.813331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.813421 (+ 90us) server_negotiation.cc:407] Connection header received
0807 22:35:27.813515 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.813516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.813528 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.813598 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.813808 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.814328 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.815395 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.815910 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.815966 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.815973 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.816180 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.816221 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:35:27.816319 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.813174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:44307 (local address 127.28.82.194:52853)
0807 22:35:27.813331 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.813344 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.813434 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.813628 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.813629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.813631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.813728 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.813730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.814399 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.814400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.815285 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.815288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.816020 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.816022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.816096 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.816101 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.816167 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.816208 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260807 22:35:27.816540 30851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "829160dfae0b427c89cc6b759a59103d" candidate_uuid: "5c2cdc81c0994ac79a2615846df2ad42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "369e4f01677b48638d033b07b9ed0a80" is_pre_election: true
I20260807 22:35:27.816627 30851 raft_consensus.cc:2454] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c2cdc81c0994ac79a2615846df2ad42 in term 0.
I20260807 22:35:27.816811 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.813503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:59743 (local address 127.28.82.195:45875)
0807 22:35:27.813566 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.813568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.813580 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:35:27.813679 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.813680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.813690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.813752 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:27.813917 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.814582 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.815816 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.816356 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.816432 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.816442 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.816692 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.816738 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:35:27.816792 30949 leader_election.cc:302] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 369e4f01677b48638d033b07b9ed0a80, 5c2cdc81c0994ac79a2615846df2ad42; no voters:
I20260807 22:35:27.816922 31366 raft_consensus.cc:2781] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:27.816952 31366 raft_consensus.cc:482] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:27.817003 31366 raft_consensus.cc:3037] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.817080 31413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.813408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:45875 (local address 127.28.82.194:59743)
0807 22:35:27.813538 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.813548 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.813613 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.813780 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.813781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.813783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:27.813849 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.813850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.814818 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.814820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.815714 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.815717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.816494 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.816495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.816596 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.816600 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.816896 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.816944 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260807 22:35:27.817319 31255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "829160dfae0b427c89cc6b759a59103d" candidate_uuid: "5c2cdc81c0994ac79a2615846df2ad42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" is_pre_election: true
I20260807 22:35:27.817417 31255 raft_consensus.cc:2454] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c2cdc81c0994ac79a2615846df2ad42 in term 0.
I20260807 22:35:27.817991 31366 raft_consensus.cc:504] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.818133 31366 leader_election.cc:288] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [CANDIDATE]: Term 1 election: Requested vote from peers 369e4f01677b48638d033b07b9ed0a80 (127.28.82.193:44307), aa29b3e1be0d411dbb6a9e273c83ade6 (127.28.82.195:45875)
I20260807 22:35:27.818279 30851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "829160dfae0b427c89cc6b759a59103d" candidate_uuid: "5c2cdc81c0994ac79a2615846df2ad42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "369e4f01677b48638d033b07b9ed0a80"
I20260807 22:35:27.818307 31255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "829160dfae0b427c89cc6b759a59103d" candidate_uuid: "5c2cdc81c0994ac79a2615846df2ad42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6"
I20260807 22:35:27.818372 30851 raft_consensus.cc:3037] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.818379 31255 raft_consensus.cc:3037] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:27.819033 30851 raft_consensus.cc:2454] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c2cdc81c0994ac79a2615846df2ad42 in term 1.
I20260807 22:35:27.819038 31255 raft_consensus.cc:2454] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c2cdc81c0994ac79a2615846df2ad42 in term 1.
I20260807 22:35:27.819197 30949 leader_election.cc:302] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 369e4f01677b48638d033b07b9ed0a80, 5c2cdc81c0994ac79a2615846df2ad42; no voters:
I20260807 22:35:27.819290 31366 raft_consensus.cc:2781] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:27.819409 31366 raft_consensus.cc:686] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 LEADER]: Becoming Leader. State: Replica: 5c2cdc81c0994ac79a2615846df2ad42, State: Running, Role: LEADER
I20260807 22:35:27.819463 31366 consensus_queue.cc:237] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } }
I20260807 22:35:27.819864 31414 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: TxnStatusTablet state changed. Reason: New leader 5c2cdc81c0994ac79a2615846df2ad42. Latest consensus state: current_term: 1 leader_uuid: "5c2cdc81c0994ac79a2615846df2ad42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.819963 31414 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:27.820132 31416 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:35:27.820325 30629 catalog_manager.cc:5495] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c2cdc81c0994ac79a2615846df2ad42 (127.28.82.194). New cstate: current_term: 1 leader_uuid: "5c2cdc81c0994ac79a2615846df2ad42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } health_report { overall_health: UNKNOWN } } }
I20260807 22:35:27.820575 31416 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:35:27.831306 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.828773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38220 (local address 127.28.82.194:33857)
0807 22:35:27.828817 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.828819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.828900 (+ 81us) server_negotiation.cc:407] Connection header received
0807 22:35:27.828998 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.828999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.829040 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.829109 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:27.829229 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.829725 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.830530 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.830731 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.830803 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.830994 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:27.831089 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:27.831166 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.831223 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.831242 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 22:35:27.831346 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.828691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:33857 (local address 127.0.0.1:38220)
0807 22:35:27.828837 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.828848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.828923 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.829118 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.829119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.829120 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:27.829168 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.829171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.829801 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.829802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.830415 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.830417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.830803 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.830804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.830872 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.830894 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:27.831151 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:27.831162 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.831217 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.831267 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 22:35:27.832504 31255 raft_consensus.cc:1261] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Refusing update from remote peer 5c2cdc81c0994ac79a2615846df2ad42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:27.832588 30851 raft_consensus.cc:1261] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Refusing update from remote peer 5c2cdc81c0994ac79a2615846df2ad42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:27.832804 31414 consensus_queue.cc:1035] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:27.832916 31366 consensus_queue.cc:1035] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:27.833564 31365 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: TxnStatusTablet state changed. Reason: New leader 5c2cdc81c0994ac79a2615846df2ad42. Latest consensus state: current_term: 1 leader_uuid: "5c2cdc81c0994ac79a2615846df2ad42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.833668 31365 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:27.833631 31384 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: TxnStatusTablet state changed. Reason: New leader 5c2cdc81c0994ac79a2615846df2ad42. Latest consensus state: current_term: 1 leader_uuid: "5c2cdc81c0994ac79a2615846df2ad42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.833731 31384 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:27.834725 31366 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c2cdc81c0994ac79a2615846df2ad42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.834851 31366 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:27.834908 31366 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5c2cdc81c0994ac79a2615846df2ad42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.834955 31366 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P 5c2cdc81c0994ac79a2615846df2ad42: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:27.835039 31374 mvcc.cc:205] Tried to move back new op lower bound from 7316038155599851520 to 7316038155549216768. Current Snapshot: MvccSnapshot[applied={T|T < 7316038155599851520 or (T in {7316038155599851520})}]
I20260807 22:35:27.835310 31372 mvcc.cc:205] Tried to move back new op lower bound from 7316038155599851520 to 7316038155549216768. Current Snapshot: MvccSnapshot[applied={T|T < 7316038155599851520}]
I20260807 22:35:27.835405 31384 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c2cdc81c0994ac79a2615846df2ad42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.835466 31384 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P 369e4f01677b48638d033b07b9ed0a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:27.836093 31365 tablet_replica.cc:439] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5c2cdc81c0994ac79a2615846df2ad42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 } } peers { permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 } } peers { permanent_uuid: "aa29b3e1be0d411dbb6a9e273c83ade6" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 45875 } } }
I20260807 22:35:27.836187 31365 tablet_replica.cc:442] T 829160dfae0b427c89cc6b759a59103d P aa29b3e1be0d411dbb6a9e273c83ade6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:27.839504 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.837047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.28.82.195:45875)
0807 22:35:27.837091 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.837093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.837102 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:35:27.837177 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.837178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.837200 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.837294 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:27.837445 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.837894 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.838738 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.838937 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.839012 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.839174 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:27.839284 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:27.839356 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.839412 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.839431 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:35:27.845330 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.841966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:33857 (local address 127.28.82.195:54463)
0807 22:35:27.842035 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.842045 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.842079 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.842723 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.842724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.842726 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:27.842781 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.842782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.843649 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.843650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.844444 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.844447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.844931 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.844932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.845011 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.845026 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:27.845201 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:27.845203 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.845238 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.845282 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 22:35:27.845328 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.842549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:54463 (local address 127.28.82.194:33857)
0807 22:35:27.842603 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.842605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.842614 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:35:27.842634 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.842635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.842644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.842703 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:27.842854 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.843561 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.844555 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.844856 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.844930 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.845106 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:27.845139 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:27.845195 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.845240 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.845260 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:35:27.848191 30851 raft_consensus.cc:1261] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Refusing update from remote peer aa29b3e1be0d411dbb6a9e273c83ade6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:27.848238 31044 raft_consensus.cc:1261] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Refusing update from remote peer aa29b3e1be0d411dbb6a9e273c83ade6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:27.848444 31396 consensus_queue.cc:1035] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "369e4f01677b48638d033b07b9ed0a80" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:27.848599 31365 consensus_queue.cc:1035] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c2cdc81c0994ac79a2615846df2ad42" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 33857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:27.858229 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.855678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:52401 (local address 127.28.82.195:45875)
0807 22:35:27.855735 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.855737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.855747 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:27.855819 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.855820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.855829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.855884 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:27.856032 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.856704 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.857482 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.857703 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.857810 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.857994 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:27.858026 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:27.858074 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.858134 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.858152 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:35:27.858778 31433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.855494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:45875 (local address 127.28.82.194:52401)
0807 22:35:27.855671 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:27.855681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:27.855753 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:27.855887 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:27.855888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:27.855890 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:27.855947 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.855948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.856796 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.856797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.857395 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:27.857397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.857814 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.857815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:27.857883 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.857902 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:27.858080 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:27.858082 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:35:27.858122 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 22:35:27.858166 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
I20260807 22:35:27.867183 30629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:27.867307 30629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:27.867949 30629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46ff06a45c2d4be18fdff1aac3527f99: Sending DeleteTablet for 3 replicas of tablet ab4ee4a206ff4963915cb0840ce4a037
I20260807 22:35:27.868203 30826 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4ee4a206ff4963915cb0840ce4a037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:27 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:51504
I20260807 22:35:27.868276 31226 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4ee4a206ff4963915cb0840ce4a037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:27 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:54322
I20260807 22:35:27.868378 31388 tablet_replica.cc:330] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: stopping tablet replica
I20260807 22:35:27.868373 31389 tablet_replica.cc:330] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: stopping tablet replica
I20260807 22:35:27.868438 31389 raft_consensus.cc:2229] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:27.868434 31388 raft_consensus.cc:2229] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:27.868470 31388 raft_consensus.cc:2258] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:27.868501 31389 raft_consensus.cc:2258] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:27.868788 31389 ts_tablet_manager.cc:1905] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:27.869309 31014 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4ee4a206ff4963915cb0840ce4a037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:27 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:38214
I20260807 22:35:27.869423 31391 tablet_replica.cc:330] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: stopping tablet replica
I20260807 22:35:27.869469 31391 raft_consensus.cc:2229] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:27.869509 31391 raft_consensus.cc:2258] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:27.869830 31391 ts_tablet_manager.cc:1905] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:27.871516 31391 ts_tablet_manager.cc:1918] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:27.871569 31391 log.cc:1196] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/ab4ee4a206ff4963915cb0840ce4a037
I20260807 22:35:27.871811 31391 ts_tablet_manager.cc:1939] T ab4ee4a206ff4963915cb0840ce4a037 P 5c2cdc81c0994ac79a2615846df2ad42: Deleting consensus metadata
I20260807 22:35:27.872149 30605 catalog_manager.cc:4841] TS 5c2cdc81c0994ac79a2615846df2ad42 (127.28.82.194:33857): tablet ab4ee4a206ff4963915cb0840ce4a037 (table test-table [id=bdbddb3c883f441abd9356cf87ec44da]) successfully deleted
I20260807 22:35:27.872375 31389 ts_tablet_manager.cc:1918] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:27.872439 31389 log.cc:1196] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/ab4ee4a206ff4963915cb0840ce4a037
I20260807 22:35:27.872665 31389 ts_tablet_manager.cc:1939] T ab4ee4a206ff4963915cb0840ce4a037 P aa29b3e1be0d411dbb6a9e273c83ade6: Deleting consensus metadata
I20260807 22:35:27.872972 30608 catalog_manager.cc:4841] TS aa29b3e1be0d411dbb6a9e273c83ade6 (127.28.82.195:45875): tablet ab4ee4a206ff4963915cb0840ce4a037 (table test-table [id=bdbddb3c883f441abd9356cf87ec44da]) successfully deleted
I20260807 22:35:27.873215 31388 ts_tablet_manager.cc:1905] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:27.874842 31388 ts_tablet_manager.cc:1918] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:27.874907 31388 log.cc:1196] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/ab4ee4a206ff4963915cb0840ce4a037
I20260807 22:35:27.875136 31388 ts_tablet_manager.cc:1939] T ab4ee4a206ff4963915cb0840ce4a037 P 369e4f01677b48638d033b07b9ed0a80: Deleting consensus metadata
I20260807 22:35:27.875473 30608 catalog_manager.cc:4841] TS 369e4f01677b48638d033b07b9ed0a80 (127.28.82.193:44307): tablet ab4ee4a206ff4963915cb0840ce4a037 (table test-table [id=bdbddb3c883f441abd9356cf87ec44da]) successfully deleted
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[30562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:27.889654 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.872256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51524 (local address 127.28.82.193:44307)
0807 22:35:27.872322 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.872324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.878663 (+ 6339us) server_negotiation.cc:407] Connection header received
0807 22:35:27.878823 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.878825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.878838 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.878906 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.879618 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.880373 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.881449 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.881759 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.881853 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.885424 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.885433 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.885433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.885441 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.887947 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.888534 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.888537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.888538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.888608 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.888875 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.888877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.888877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.889072 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.889218 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.889461 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.889553 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 22:35:27.890246 30910 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51524
I20260807 22:35:27.898963 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:27.891282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.28.82.254:43905)
0807 22:35:27.891328 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:27.891330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:27.891549 (+ 219us) server_negotiation.cc:407] Connection header received
0807 22:35:27.891590 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:27.891591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:27.891603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:27.891665 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:27.892278 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.893033 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.894053 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:27.894368 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:27.894436 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:27.895901 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:27.895903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:27.895903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:27.895911 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:27.897579 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.898036 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.898037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.898037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.898108 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:27.898384 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:27.898385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:27.898385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:27.898553 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:27.898655 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:27.898791 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 22:35:27.898870 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 22:35:27.899260 30629 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40496
I20260807 22:35:27.899856 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 30727
I20260807 22:35:27.913456 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 30925
I20260807 22:35:27.930178 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 31130
I20260807 22:35:27.936959 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 30582
2026-08-07T22:35:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2556 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[31450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[31450](info): set up 2 sockets
Aug 07 22:35:27 dist-test-slave-7kzw krb5kdc[31450](info): commencing operation
krb5kdc: starting...
W20260807 22:35:29.159348 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:29.279537 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:44091
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:44091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:29.363500 31475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:29.363703 31475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:29.363725 31475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:29.364897 31475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:29.364939 31475 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:35:29.364953 31475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:29.364966 31475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:29.364979 31475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:29.364990 31475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:29.366401 31475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:44091
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:44091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:29.366673 31475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:29.366894 31475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:29.371659 31480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:29.371637 31481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:29.371637 31487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:29.372092 31475 server_base.cc:1034] running on GCE node
I20260807 22:35:29.372370 31475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:29.372656 31475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:29.373839 31475 hybrid_clock.cc:648] HybridClock initialized: now 1786142129373824 us; error 28 us; skew 500 ppm
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:29.379112 31475 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:29.379577 31475 webserver.cc:460] Webserver started at http://127.28.82.254:38297/ using document root <none> and password file <none>
I20260807 22:35:29.379824 31475 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:29.379873 31475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:29.380003 31475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:29.381068 31475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "2a08dbc76e9e4b79a3628c3611928625"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "342a6e70af6b3e30357920683f007d2f"
server_key_iv: "80257661d30d62d0c7dc945abf4c7e97"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.381454 31475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "2a08dbc76e9e4b79a3628c3611928625"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "342a6e70af6b3e30357920683f007d2f"
server_key_iv: "80257661d30d62d0c7dc945abf4c7e97"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.382839 31475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:29.383608 31495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:29.383780 31475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:29.383885 31475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "2a08dbc76e9e4b79a3628c3611928625"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "342a6e70af6b3e30357920683f007d2f"
server_key_iv: "80257661d30d62d0c7dc945abf4c7e97"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.383956 31475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:29.411129 31475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:29.413197 31475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:29.413364 31475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:29.418272 31475 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:44091
I20260807 22:35:29.418326 31580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:44091 every 8 connection(s)
I20260807 22:35:29.418715 31475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:29.419394 31581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:29.421672 31581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: Bootstrap starting.
I20260807 22:35:29.422243 31581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:29.422504 31581 log.cc:824] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:29.423236 31581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: No bootstrap required, opened a new log
I20260807 22:35:29.424736 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 31475
I20260807 22:35:29.424844 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:29.425047 29003 external_mini_cluster.cc:1428] Setting key 1e00445a8541141a1f530a42152a5705
I20260807 22:35:29.425958 31581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a08dbc76e9e4b79a3628c3611928625" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44091 } }
I20260807 22:35:29.426100 31581 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:29.426195 31581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a08dbc76e9e4b79a3628c3611928625, State: Initialized, Role: FOLLOWER
I20260807 22:35:29.426362 31581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a08dbc76e9e4b79a3628c3611928625" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44091 } }
I20260807 22:35:29.426486 31581 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:29.426533 31581 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:29.426589 31581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:29.427309 31581 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a08dbc76e9e4b79a3628c3611928625" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44091 } }
I20260807 22:35:29.427433 31581 leader_election.cc:302] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a08dbc76e9e4b79a3628c3611928625; no voters:
I20260807 22:35:29.427606 31581 leader_election.cc:288] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:29.427650 31592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:29.427865 31592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [term 1 LEADER]: Becoming Leader. State: Replica: 2a08dbc76e9e4b79a3628c3611928625, State: Running, Role: LEADER
I20260807 22:35:29.427872 31581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:29.428025 31592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a08dbc76e9e4b79a3628c3611928625" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44091 } }
I20260807 22:35:29.429332 31593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a08dbc76e9e4b79a3628c3611928625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a08dbc76e9e4b79a3628c3611928625" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44091 } } }
I20260807 22:35:29.429433 31593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:29.429459 31596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a08dbc76e9e4b79a3628c3611928625. Latest consensus state: current_term: 1 leader_uuid: "2a08dbc76e9e4b79a3628c3611928625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a08dbc76e9e4b79a3628c3611928625" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44091 } } }
I20260807 22:35:29.429515 31596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:29.429653 31606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:29.430186 31606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:29.432085 31606 catalog_manager.cc:1347] Generated new cluster ID: 661db2586bd84175955c08efbb7ebcf4
I20260807 22:35:29.432140 31606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:29.436906 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.425639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.28.82.254:44091)
0807 22:35:29.425853 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.425861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.425886 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:35:29.425982 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.425987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.426029 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.426185 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:29.427003 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.428252 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.429376 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.430073 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.430191 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.433095 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:29.433099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:29.433099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:29.433111 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:29.435154 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.435528 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.435531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.435532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.435696 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.435904 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.435906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.435906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.436219 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:29.436419 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.436506 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.436685 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260807 22:35:29.446580 31606 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:29.447048 31606 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:29.451781 31606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: Generated new TSK 0
I20260807 22:35:29.451954 31606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:29.484546 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44091
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:29.573897 31624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:29.574110 31624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:29.574136 31624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:29.575356 31624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:29.575400 31624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:29.575421 31624 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:35:29.575433 31624 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:29.575450 31624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:29.576920 31624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:29.577195 31624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:29.577432 31624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:35:29.582595 31624 server_base.cc:1034] running on GCE node
W20260807 22:35:29.582578 31631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:29.582695 31635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:29.582578 31632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:29.583027 31624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:29.583281 31624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:29.584451 31624 hybrid_clock.cc:648] HybridClock initialized: now 1786142129584423 us; error 52 us; skew 500 ppm
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:29.587097 31624 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:29.587522 31624 webserver.cc:460] Webserver started at http://127.28.82.193:42027/ using document root <none> and password file <none>
I20260807 22:35:29.587778 31624 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:29.587844 31624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:29.587989 31624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:29.588985 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "a238c20aed7f413b9efd60a30b0a7599"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "866630b8f66543324536146437621c87"
server_key_iv: "990d0ca286e4fd0cc8c15a501672f94e"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.589347 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "a238c20aed7f413b9efd60a30b0a7599"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "866630b8f66543324536146437621c87"
server_key_iv: "990d0ca286e4fd0cc8c15a501672f94e"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.590698 31624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:35:29.591392 31644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:29.591565 31624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:29.591665 31624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "a238c20aed7f413b9efd60a30b0a7599"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "866630b8f66543324536146437621c87"
server_key_iv: "990d0ca286e4fd0cc8c15a501672f94e"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.591738 31624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:29.620656 31624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:29.622967 31624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:29.623127 31624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:29.623674 31624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:29.623737 31624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:29.623775 31624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:29.623816 31624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:29.631336 31624 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:44565
I20260807 22:35:29.631384 31807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:44565 every 8 connection(s)
I20260807 22:35:29.631690 31624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:29.633829 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.624136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:60013 (local address 127.28.82.254:44091)
0807 22:35:29.624209 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.624212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.624432 (+ 220us) server_negotiation.cc:407] Connection header received
0807 22:35:29.624940 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.624941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.624955 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.625031 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:29.625974 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.626857 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.627863 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.628209 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.628277 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.631120 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:29.631122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:29.631122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:29.631135 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:29.632757 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.633104 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.633104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.633105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.633160 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.633332 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.633333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.633333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.633431 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:29.633481 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.633698 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.633764 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:35:29.633991 31656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.624114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44091 (local address 127.28.82.193:60013)
0807 22:35:29.624294 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.624333 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.624837 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.625107 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.625111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.625511 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:29.625855 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.625863 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.626961 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.626962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.627766 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.627768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.628316 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.628317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.628460 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.629041 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:29.629050 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:29.630999 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:29.632825 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.632830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.632832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.633040 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.633200 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.633200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.633200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.633286 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.633484 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:29.633490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:29.633594 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.633671 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.633806 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
I20260807 22:35:29.634549 31808 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44091
I20260807 22:35:29.634653 31808 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:29.634827 31808 heartbeater.cc:511] Master 127.28.82.254:44091 requested a full tablet report, sending...
I20260807 22:35:29.635385 31517 ts_manager.cc:194] Registered new tserver with Master: a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565)
I20260807 22:35:29.636297 31517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:60013
I20260807 22:35:29.640568 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 31624
I20260807 22:35:29.640681 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:29.640879 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.635585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:44135 (local address 127.28.82.254:44091)
0807 22:35:29.635627 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.635629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.635762 (+ 133us) server_negotiation.cc:407] Connection header received
0807 22:35:29.635829 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.635830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.635838 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.635881 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:29.636183 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.636796 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.637482 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.637692 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.637762 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.638601 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:29.638602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:29.638603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:29.638610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:29.639725 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.640053 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.640054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.640054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.640110 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.640341 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.640342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.640343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.640469 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:29.640541 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.640707 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.640790 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:35:29.640897 29003 external_mini_cluster.cc:1428] Setting key ac4c1a92dc4f69186f1c3e4e1d4836ad
I20260807 22:35:29.640899 31816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.635531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44091 (local address 127.28.82.193:44135)
0807 22:35:29.635692 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.635703 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.635775 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.635886 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.635886 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.636062 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:29.636125 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.636126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.636887 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.636888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.637399 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.637401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.637773 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.637774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.637851 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.638188 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:29.638192 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:29.638514 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:29.639808 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.639809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.639809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.639982 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.640199 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.640199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.640200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.640265 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.640532 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:29.640533 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:29.640596 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.640679 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.640802 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:29.667039 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44091
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:29.748364 31819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:29.748591 31819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:29.748638 31819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:29.749761 31819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:29.749825 31819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:29.749857 31819 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:35:29.749884 31819 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:29.749919 31819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:29.751758 31819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:29.752009 31819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:29.752249 31819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:29.757093 31825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:29.757246 31819 server_base.cc:1034] running on GCE node
W20260807 22:35:29.757093 31828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:29.757123 31826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:29.757597 31819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:29.757874 31819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:29.759027 31819 hybrid_clock.cc:648] HybridClock initialized: now 1786142129759013 us; error 26 us; skew 500 ppm
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:29.761600 31819 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:29.761981 31819 webserver.cc:460] Webserver started at http://127.28.82.194:38193/ using document root <none> and password file <none>
I20260807 22:35:29.762226 31819 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:29.762271 31819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:29.762367 31819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:29.763373 31819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "4e0741339bf644689b165ba0fd83e669"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "31a8d2b72eb09ed183bb14bbb7579406"
server_key_iv: "6dcd294e8a0c743443002065142573b9"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.763688 31819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "4e0741339bf644689b165ba0fd83e669"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "31a8d2b72eb09ed183bb14bbb7579406"
server_key_iv: "6dcd294e8a0c743443002065142573b9"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.764941 31819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:29.765618 31840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:29.765780 31819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:29.765872 31819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "4e0741339bf644689b165ba0fd83e669"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "31a8d2b72eb09ed183bb14bbb7579406"
server_key_iv: "6dcd294e8a0c743443002065142573b9"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.765946 31819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:29.774454 31819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:29.776252 31819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:29.776407 31819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:29.776901 31819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:29.776964 31819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:29.777007 31819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:29.777031 31819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:29.784128 31819 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:42977
I20260807 22:35:29.784436 32012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:42977 every 8 connection(s)
I20260807 22:35:29.784528 31819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:29.788079 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.777376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:35591 (local address 127.28.82.254:44091)
0807 22:35:29.777473 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.777476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.777625 (+ 149us) server_negotiation.cc:407] Connection header received
0807 22:35:29.778071 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.778072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.778087 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.778159 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:29.778948 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.779750 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.781425 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.781780 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.781858 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.784718 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:29.784720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:29.784720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:29.784729 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:29.786545 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.787081 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.787082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.787083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.787166 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.787477 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.787478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.787478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.787642 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:29.787731 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.787947 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.788013 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:35:29.788232 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.777343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44091 (local address 127.28.82.194:35591)
0807 22:35:29.777494 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.777533 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.777967 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.778215 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.778219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.778568 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:29.778821 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.778828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.779859 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.779860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.781328 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.781331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.781888 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.781889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.782053 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.783002 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:29.783011 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:29.784589 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:29.786630 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.786636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.786638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.786996 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.787249 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.787250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.787250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.787399 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.787723 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:29.787728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:29.787842 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.787928 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.788042 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260807 22:35:29.788692 32013 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44091
I20260807 22:35:29.788781 32013 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:29.788976 32013 heartbeater.cc:511] Master 127.28.82.254:44091 requested a full tablet report, sending...
I20260807 22:35:29.789373 31518 ts_manager.cc:194] Registered new tserver with Master: 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977)
I20260807 22:35:29.789839 31518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:35591
I20260807 22:35:29.791818 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 31819
I20260807 22:35:29.791926 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:29.792179 29003 external_mini_cluster.cc:1428] Setting key 1b82f89d049ab4fba9913e919d7dbe2c
I20260807 22:35:29.796144 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.789824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:43629 (local address 127.28.82.254:44091)
0807 22:35:29.789877 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.789879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.789920 (+ 41us) server_negotiation.cc:407] Connection header received
0807 22:35:29.790020 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.790021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.790032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.790086 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:29.790536 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.791050 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.791700 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.792029 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.792114 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.793188 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:29.793190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:29.793190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:29.793196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:29.794752 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.795218 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.795219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.795219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.795288 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.795522 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.795523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.795524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.795683 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:29.795765 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.795952 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.796034 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 22:35:29.796152 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.789701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44091 (local address 127.28.82.194:43629)
0807 22:35:29.789856 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.789868 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.789933 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.790093 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.790094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.790309 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:29.790370 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.790372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.791151 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.791152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.791619 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.791620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.792110 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.792111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.792181 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.792614 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:29.792619 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:29.793096 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:29.794831 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.794833 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.794833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.795112 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.795382 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.795383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.795383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.795449 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.795768 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:29.795769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:29.795821 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.795921 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.796035 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:29.818964 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44091
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:29.901039 32025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:29.901269 32025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:29.901311 32025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:29.902451 32025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:29.902542 32025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:29.902576 32025 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 22:35:29.902606 32025 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:29.902637 32025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:29.904098 32025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:29.904361 32025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:29.904600 32025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:29.909487 32037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:29.909500 32033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:29.909492 32034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:29.909821 32025 server_base.cc:1034] running on GCE node
I20260807 22:35:29.910015 32025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:29.910248 32025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:29.911430 32025 hybrid_clock.cc:648] HybridClock initialized: now 1786142129911417 us; error 38 us; skew 500 ppm
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:29.914031 32025 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:29.914597 32025 webserver.cc:460] Webserver started at http://127.28.82.195:39883/ using document root <none> and password file <none>
I20260807 22:35:29.914812 32025 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:29.914855 32025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:29.914944 32025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:29.915895 32025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "9e03f1a6ccf3462dbad7e5ea6892da94"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "0bfdf5464aff8b2747babfdfacb3fe35"
server_key_iv: "317c6a420db81d110f0fd14bf6ecf3f4"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.916208 32025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "9e03f1a6ccf3462dbad7e5ea6892da94"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "0bfdf5464aff8b2747babfdfacb3fe35"
server_key_iv: "317c6a420db81d110f0fd14bf6ecf3f4"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.917529 32025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:29.918262 32048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:29.918445 32025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:29.918540 32025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "9e03f1a6ccf3462dbad7e5ea6892da94"
format_stamp: "Formatted at 2026-08-07 22:35:29 on dist-test-slave-7kzw"
server_key: "0bfdf5464aff8b2747babfdfacb3fe35"
server_key_iv: "317c6a420db81d110f0fd14bf6ecf3f4"
server_key_version: "encryptionkey@0"
I20260807 22:35:29.918596 32025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:29.935726 32025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:29.937799 32025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:29.937911 32025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:29.938366 32025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:29.938405 32025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:29.938429 32025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:29.938493 32025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:29.945111 32025 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:38897
I20260807 22:35:29.945191 32222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:38897 every 8 connection(s)
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:29.945592 32025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:29.948730 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.938875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:41741 (local address 127.28.82.254:44091)
0807 22:35:29.938944 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.938947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.939133 (+ 186us) server_negotiation.cc:407] Connection header received
0807 22:35:29.939604 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.939606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.939620 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.939692 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:29.940560 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.941416 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.942955 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.943316 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.943403 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.946087 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:29.946089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:29.946089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:29.946096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:29.947467 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.947885 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.947886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.947887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.947956 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.948207 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.948208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.948208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.948329 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:29.948390 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.948598 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.948663 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:35:29.948909 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.938841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44091 (local address 127.28.82.195:41741)
0807 22:35:29.938995 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.939047 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.939504 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.939805 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.939809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.940174 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:29.940449 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.940456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.941516 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.941517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.942866 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.942868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.943402 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.943403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.943497 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.944030 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:29.944047 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:29.945981 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:29.947541 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.947547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.947549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.947794 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.948026 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.948026 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.948026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.948127 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.948390 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:29.948393 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:29.948498 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.948577 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.948709 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260807 22:35:29.949343 32224 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44091
I20260807 22:35:29.949422 32224 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:29.949620 32224 heartbeater.cc:511] Master 127.28.82.254:44091 requested a full tablet report, sending...
I20260807 22:35:29.950050 31517 ts_manager.cc:194] Registered new tserver with Master: 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897)
I20260807 22:35:29.950552 31517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:41741
I20260807 22:35:29.953977 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 32025
I20260807 22:35:29.954078 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:29.954265 29003 external_mini_cluster.cc:1428] Setting key 21d7df6c60d5a10d6d9095f58699d41f
I20260807 22:35:29.955080 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:35:29.956331 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.950404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44091 (local address 127.28.82.195:36891)
0807 22:35:29.950554 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.950566 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.950632 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.950812 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.950813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.950994 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:29.951056 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.951057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.951700 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.951701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.952199 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.952200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.952566 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.952567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.952641 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.953001 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:29.953005 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:29.953273 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:29.955304 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.955305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.955305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.955498 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.955707 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:29.955707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:29.955708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:29.955756 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:29.956054 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:29.956055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:29.956104 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.956169 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.956252 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260807 22:35:29.956429 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.950454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:36891 (local address 127.28.82.254:44091)
0807 22:35:29.950523 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.950525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.950623 (+ 98us) server_negotiation.cc:407] Connection header received
0807 22:35:29.950742 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.950743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.950751 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.950789 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:29.951137 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.951600 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.952281 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.952481 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.952564 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.953359 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:29.953361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:29.953361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:29.953369 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:29.955238 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.955576 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.955577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.955578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.955644 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.955840 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.955841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.955841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.955980 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:29.956056 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.956211 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.956294 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:35:29 dist-test-slave-7kzw krb5kdc[31450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:29.980525 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.973244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.28.82.254:44091)
0807 22:35:29.973305 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.973307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.973315 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:29.973339 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.973340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.973349 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.973400 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:29.973936 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.974423 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.975197 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.975378 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.975436 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.977629 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:29.977630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:29.977631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:29.977637 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:29.979080 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.979566 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.979567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.979568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.979657 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:29.979853 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:29.979854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:29.979855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:29.979965 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:29.980061 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.980310 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.980422 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:35:29.982293 31517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:29.983999 31517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:29.991219 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.986984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35916 (local address 127.28.82.193:44565)
0807 22:35:29.987176 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.987180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.987553 (+ 373us) server_negotiation.cc:407] Connection header received
0807 22:35:29.987603 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.987605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.987617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.987695 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:29.987831 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.988570 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.990124 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.990870 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.990938 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.990986 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.991062 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.991100 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
I20260807 22:35:29.991233 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.986564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38897 (local address 127.0.0.1:51894)
0807 22:35:29.986746 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.986802 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.986871 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.987375 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.987376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.987378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:29.987438 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.987440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.988752 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.988754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.989901 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.989905 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.990941 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.990942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.991008 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.991012 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.991062 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.991104 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":77,"thread_start_us":69,"threads_started":1}
I20260807 22:35:29.991110 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.986566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:42977 (local address 127.0.0.1:40610)
0807 22:35:29.986850 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.986860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.986915 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.987311 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.987314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.987317 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:29.987386 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.987389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.988734 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.988735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.990055 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.990057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.990788 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.990789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.990880 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.990894 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.990963 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.991015 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":70,"thread_start_us":94,"threads_started":1}
I20260807 22:35:29.991178 32243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.986564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:44565 (local address 127.0.0.1:35916)
0807 22:35:29.987439 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:29.987450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:29.987497 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:29.987692 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:29.987693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:29.987695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:29.987730 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.987732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.988693 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.988694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.990035 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:29.990039 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.990936 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.990936 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:29.990995 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.990999 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:29.991051 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 22:35:29.991086 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":48,"thread_start_us":85,"threads_started":1}
I20260807 22:35:29.991297 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.986842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.28.82.194:42977)
0807 22:35:29.987048 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.987052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.987060 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:29.987117 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.987119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.987135 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.987217 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:29.987470 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.988625 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.990143 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.990706 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.990789 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.990848 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.991154 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.991197 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":102,"threads_started":1}
I20260807 22:35:29.991765 32250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:29.986771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51894 (local address 127.28.82.195:38897)
0807 22:35:29.987201 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:29.987206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:29.987217 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:35:29.987259 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:29.987261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:29.987283 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:29.987362 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:29.987508 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.988626 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.989992 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:29.990772 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:29.990845 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:29.990902 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:29.991221 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 22:35:29.991501 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":96,"threads_started":1}
I20260807 22:35:29.992158 31911 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb24fd57a9b4059b102772565d9d827 (DEFAULT_TABLE table=test-table [id=2c1ebc57eac54986a492da19f43ffb36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:29.992164 31716 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb24fd57a9b4059b102772565d9d827 (DEFAULT_TABLE table=test-table [id=2c1ebc57eac54986a492da19f43ffb36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:29.992497 31716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb24fd57a9b4059b102772565d9d827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:29.992499 31911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb24fd57a9b4059b102772565d9d827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:29.995584 32255 tablet_bootstrap.cc:492] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Bootstrap starting.
I20260807 22:35:29.995751 32256 tablet_bootstrap.cc:492] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Bootstrap starting.
I20260807 22:35:29.996060 32122 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb24fd57a9b4059b102772565d9d827 (DEFAULT_TABLE table=test-table [id=2c1ebc57eac54986a492da19f43ffb36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:29.996275 32255 tablet_bootstrap.cc:654] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:29.996367 32256 tablet_bootstrap.cc:654] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:29.996436 32122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb24fd57a9b4059b102772565d9d827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:29.996613 32255 log.cc:824] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:29.996717 32256 log.cc:824] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:29.997445 32255 tablet_bootstrap.cc:492] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: No bootstrap required, opened a new log
I20260807 22:35:29.997522 32255 ts_tablet_manager.cc:1397] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:29.999650 32255 raft_consensus.cc:348] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:29.999830 32255 raft_consensus.cc:374] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:29.999720 32259 tablet_bootstrap.cc:492] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Bootstrap starting.
I20260807 22:35:29.999881 32255 raft_consensus.cc:729] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0741339bf644689b165ba0fd83e669, State: Initialized, Role: FOLLOWER
I20260807 22:35:29.999995 32255 consensus_queue.cc:260] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:30.000312 32255 ts_tablet_manager.cc:1428] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:30.000370 32013 heartbeater.cc:503] Master 127.28.82.254:44091 was elected leader, sending a full tablet report...
I20260807 22:35:30.000849 32259 tablet_bootstrap.cc:654] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:30.000876 32256 tablet_bootstrap.cc:492] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: No bootstrap required, opened a new log
I20260807 22:35:30.000963 32256 ts_tablet_manager.cc:1397] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 22:35:30.001223 32259 log.cc:824] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:30.001987 32259 tablet_bootstrap.cc:492] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: No bootstrap required, opened a new log
I20260807 22:35:30.002058 32259 ts_tablet_manager.cc:1397] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:35:30.002871 32256 raft_consensus.cc:348] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:30.003019 32256 raft_consensus.cc:374] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:30.003067 32256 raft_consensus.cc:729] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a238c20aed7f413b9efd60a30b0a7599, State: Initialized, Role: FOLLOWER
I20260807 22:35:30.003187 32256 consensus_queue.cc:260] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:30.003439 32256 ts_tablet_manager.cc:1428] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 22:35:30.003520 31808 heartbeater.cc:503] Master 127.28.82.254:44091 was elected leader, sending a full tablet report...
I20260807 22:35:30.004022 32259 raft_consensus.cc:348] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:30.004195 32259 raft_consensus.cc:374] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:30.004251 32259 raft_consensus.cc:729] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e03f1a6ccf3462dbad7e5ea6892da94, State: Initialized, Role: FOLLOWER
I20260807 22:35:30.004379 32259 consensus_queue.cc:260] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:30.004704 32259 ts_tablet_manager.cc:1428] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:30.004732 32224 heartbeater.cc:503] Master 127.28.82.254:44091 was elected leader, sending a full tablet report...
W20260807 22:35:30.035320 32015 tablet.cc:2369] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:35:30.132401 31810 tablet.cc:2369] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:35:30.196144 32225 tablet.cc:2369] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:30.197214 32265 raft_consensus.cc:482] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:30.197320 32265 raft_consensus.cc:504] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:30.197665 32265 leader_election.cc:288] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565), 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977)
I20260807 22:35:30.200497 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.197884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:44565 (local address 127.28.82.195:57009)
0807 22:35:30.197945 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.197957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.198007 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.198195 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.198196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.198198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:30.198265 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.198267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.198969 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.198970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.199708 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.199710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.200235 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.200236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.200320 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.200323 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.200387 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.200425 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 22:35:30.200497 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.197913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:57009 (local address 127.28.82.193:44565)
0807 22:35:30.197991 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.197994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.198009 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:35:30.198081 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.198082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.198092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.198167 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:30.198337 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.198876 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.199803 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.200149 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.200237 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.200246 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.200395 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.200429 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:35:30.200659 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.198081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:43055 (local address 127.28.82.194:42977)
0807 22:35:30.198145 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.198148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.198234 (+ 86us) server_negotiation.cc:407] Connection header received
0807 22:35:30.198325 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.198326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.198336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.198403 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:30.198588 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.199059 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.199937 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.200284 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.200366 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.200374 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.200525 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.200566 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:35:30.200863 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.197980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:42977 (local address 127.28.82.195:43055)
0807 22:35:30.198163 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.198173 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.198251 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.198405 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.198406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.198407 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:30.198513 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.198515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.199145 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.199146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.199845 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.199846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.200366 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.200367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.200444 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.200452 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.200518 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.200706 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":126,"threads_started":1}
I20260807 22:35:30.200825 31742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb24fd57a9b4059b102772565d9d827" candidate_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a238c20aed7f413b9efd60a30b0a7599" is_pre_election: true
I20260807 22:35:30.201005 31742 raft_consensus.cc:2454] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e03f1a6ccf3462dbad7e5ea6892da94 in term 0.
I20260807 22:35:30.201237 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb24fd57a9b4059b102772565d9d827" candidate_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669" is_pre_election: true
I20260807 22:35:30.201262 32049 leader_election.cc:302] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e03f1a6ccf3462dbad7e5ea6892da94, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:30.201377 31939 raft_consensus.cc:2454] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e03f1a6ccf3462dbad7e5ea6892da94 in term 0.
I20260807 22:35:30.201467 32265 raft_consensus.cc:2781] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:30.201539 32265 raft_consensus.cc:482] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:30.201565 32265 raft_consensus.cc:3037] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.202296 32265 raft_consensus.cc:504] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:30.202541 32265 leader_election.cc:288] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [CANDIDATE]: Term 1 election: Requested vote from peers a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565), 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977)
I20260807 22:35:30.202633 31742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb24fd57a9b4059b102772565d9d827" candidate_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a238c20aed7f413b9efd60a30b0a7599"
I20260807 22:35:30.202723 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb24fd57a9b4059b102772565d9d827" candidate_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669"
I20260807 22:35:30.202737 31742 raft_consensus.cc:3037] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.202809 31939 raft_consensus.cc:3037] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.203490 31939 raft_consensus.cc:2454] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e03f1a6ccf3462dbad7e5ea6892da94 in term 1.
I20260807 22:35:30.203564 31742 raft_consensus.cc:2454] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e03f1a6ccf3462dbad7e5ea6892da94 in term 1.
I20260807 22:35:30.203817 32049 leader_election.cc:302] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e03f1a6ccf3462dbad7e5ea6892da94, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:30.203967 32265 raft_consensus.cc:2781] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:30.204188 32265 raft_consensus.cc:686] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 LEADER]: Becoming Leader. State: Replica: 9e03f1a6ccf3462dbad7e5ea6892da94, State: Running, Role: LEADER
I20260807 22:35:30.204301 32265 consensus_queue.cc:237] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } }
I20260807 22:35:30.205314 31517 catalog_manager.cc:5495] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195). New cstate: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:35:30 dist-test-slave-7kzw krb5kdc[31450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142129, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:30.234882 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.227232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44091 (local address 127.0.0.1:47526)
0807 22:35:30.227446 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.227456 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.227541 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.227718 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.227719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.228038 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:30.228086 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.228088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.228841 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.228842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.229557 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.229560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.229955 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.229956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.230025 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.230657 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:30.230662 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:30.232173 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:30.233712 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:30.233720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:30.233722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:30.233937 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:30.234155 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:30.234156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:30.234156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:30.234243 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:30.234609 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:30.234615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:30.234659 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.234717 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.234806 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":131,"threads_started":1}
I20260807 22:35:30.234954 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.227315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.28.82.254:44091)
0807 22:35:30.227402 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.227405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.227531 (+ 126us) server_negotiation.cc:407] Connection header received
0807 22:35:30.227593 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.227593 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.227602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.227683 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:30.228154 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.228749 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.229670 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.229873 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.229944 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.232262 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:30.232264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:30.232264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:30.232271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:30.233607 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:30.234031 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:30.234032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:30.234032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:30.234099 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:30.234309 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:30.234310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:30.234311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:30.234443 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:30.234613 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.234755 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.234860 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":39}
I20260807 22:35:30.237126 31517 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:47526:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:35:30.237313 31517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:30.238957 31716 tablet_service.cc:1467] Processing CreateTablet for tablet 5744c143f0a04d08b6a3113892298f43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0a0ef3359cfa4f1bb2a8cc1fc6dd1cd2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:30.239028 31911 tablet_service.cc:1467] Processing CreateTablet for tablet 5744c143f0a04d08b6a3113892298f43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0a0ef3359cfa4f1bb2a8cc1fc6dd1cd2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:30.239161 31716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5744c143f0a04d08b6a3113892298f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:30.239168 31911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5744c143f0a04d08b6a3113892298f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:30.239210 32122 tablet_service.cc:1467] Processing CreateTablet for tablet 5744c143f0a04d08b6a3113892298f43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0a0ef3359cfa4f1bb2a8cc1fc6dd1cd2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:30.239385 32122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5744c143f0a04d08b6a3113892298f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:30.240828 32255 tablet_bootstrap.cc:492] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: Bootstrap starting.
I20260807 22:35:30.240900 32259 tablet_bootstrap.cc:492] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: Bootstrap starting.
I20260807 22:35:30.241271 32259 tablet_bootstrap.cc:654] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:30.241447 32255 tablet_bootstrap.cc:654] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:30.241806 32256 tablet_bootstrap.cc:492] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: Bootstrap starting.
I20260807 22:35:30.241891 32259 tablet_bootstrap.cc:492] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: No bootstrap required, opened a new log
I20260807 22:35:30.241926 32259 ts_tablet_manager.cc:1397] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:30.242053 32259 raft_consensus.cc:348] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.242100 32259 raft_consensus.cc:374] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:30.242112 32259 raft_consensus.cc:729] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e03f1a6ccf3462dbad7e5ea6892da94, State: Initialized, Role: FOLLOWER
I20260807 22:35:30.242147 32259 consensus_queue.cc:260] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.242216 32256 tablet_bootstrap.cc:654] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:30.242246 32259 ts_tablet_manager.cc:1428] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:30.242288 32265 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.242374 32265 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:30.242699 32255 tablet_bootstrap.cc:492] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: No bootstrap required, opened a new log
I20260807 22:35:30.242762 32255 ts_tablet_manager.cc:1397] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:30.242946 32256 tablet_bootstrap.cc:492] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: No bootstrap required, opened a new log
I20260807 22:35:30.242933 32255 raft_consensus.cc:348] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.242995 32256 ts_tablet_manager.cc:1397] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:30.243006 32255 raft_consensus.cc:374] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:30.243032 32255 raft_consensus.cc:729] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0741339bf644689b165ba0fd83e669, State: Initialized, Role: FOLLOWER
I20260807 22:35:30.243088 32255 consensus_queue.cc:260] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.243201 32255 ts_tablet_manager.cc:1428] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:30.243166 32256 raft_consensus.cc:348] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.243242 32256 raft_consensus.cc:374] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:30.243268 32256 raft_consensus.cc:729] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a238c20aed7f413b9efd60a30b0a7599, State: Initialized, Role: FOLLOWER
I20260807 22:35:30.243319 32256 consensus_queue.cc:260] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.243351 32261 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.243453 32256 ts_tablet_manager.cc:1428] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:30.243453 32261 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:30.243580 32264 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.243680 32264 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:30.374655 32265 raft_consensus.cc:482] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:30.374817 32265 raft_consensus.cc:504] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.375020 32265 leader_election.cc:288] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565), 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977)
I20260807 22:35:30.375277 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5744c143f0a04d08b6a3113892298f43" candidate_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669" is_pre_election: true
I20260807 22:35:30.375277 31742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5744c143f0a04d08b6a3113892298f43" candidate_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a238c20aed7f413b9efd60a30b0a7599" is_pre_election: true
I20260807 22:35:30.375396 31742 raft_consensus.cc:2454] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e03f1a6ccf3462dbad7e5ea6892da94 in term 0.
I20260807 22:35:30.375396 31939 raft_consensus.cc:2454] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e03f1a6ccf3462dbad7e5ea6892da94 in term 0.
I20260807 22:35:30.375560 32049 leader_election.cc:302] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4e0741339bf644689b165ba0fd83e669, 9e03f1a6ccf3462dbad7e5ea6892da94, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:30.375706 32265 raft_consensus.cc:2781] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:30.375742 32265 raft_consensus.cc:482] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:30.375773 32265 raft_consensus.cc:3037] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.376482 32265 raft_consensus.cc:504] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.376637 32265 leader_election.cc:288] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [CANDIDATE]: Term 1 election: Requested vote from peers a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565), 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977)
I20260807 22:35:30.376793 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5744c143f0a04d08b6a3113892298f43" candidate_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669"
I20260807 22:35:30.376875 31939 raft_consensus.cc:3037] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.376820 31742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5744c143f0a04d08b6a3113892298f43" candidate_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a238c20aed7f413b9efd60a30b0a7599"
I20260807 22:35:30.376914 31742 raft_consensus.cc:3037] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.377502 31939 raft_consensus.cc:2454] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e03f1a6ccf3462dbad7e5ea6892da94 in term 1.
I20260807 22:35:30.377575 31742 raft_consensus.cc:2454] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e03f1a6ccf3462dbad7e5ea6892da94 in term 1.
I20260807 22:35:30.377694 32052 leader_election.cc:302] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0741339bf644689b165ba0fd83e669, 9e03f1a6ccf3462dbad7e5ea6892da94; no voters:
I20260807 22:35:30.377796 32265 raft_consensus.cc:2781] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:30.377863 32265 raft_consensus.cc:686] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 LEADER]: Becoming Leader. State: Replica: 9e03f1a6ccf3462dbad7e5ea6892da94, State: Running, Role: LEADER
I20260807 22:35:30.377923 32265 consensus_queue.cc:237] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.378271 32268 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: TxnStatusTablet state changed. Reason: New leader 9e03f1a6ccf3462dbad7e5ea6892da94. Latest consensus state: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.378376 32268 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:30.378603 32286 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:35:30.378767 32286 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:35:30.378649 31517 catalog_manager.cc:5495] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195). New cstate: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } health_report { overall_health: HEALTHY } } }
I20260807 22:35:30.388748 32250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.385815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.28.82.195:38897)
0807 22:35:30.385864 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.385866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.385897 (+ 31us) server_negotiation.cc:407] Connection header received
0807 22:35:30.385987 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.385988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.386003 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.386060 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:30.386217 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.386799 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.387952 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.388141 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.388194 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.388391 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.388513 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.388602 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.388661 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.388682 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 22:35:30.388833 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.385757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38897 (local address 127.0.0.1:51900)
0807 22:35:30.385845 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.385856 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.385912 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.386084 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.386086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.386087 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:30.386148 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.386151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.386884 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.386885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.387845 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.387848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.388216 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.388217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.388287 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.388307 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.388584 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.388596 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.388654 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.388747 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 22:35:30.390568 31939 raft_consensus.cc:1261] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Refusing update from remote peer 9e03f1a6ccf3462dbad7e5ea6892da94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:30.390626 31742 raft_consensus.cc:1261] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Refusing update from remote peer 9e03f1a6ccf3462dbad7e5ea6892da94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:30.390887 32265 consensus_queue.cc:1035] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:30.391021 32268 consensus_queue.cc:1035] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:30.392619 32261 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: TxnStatusTablet state changed. Reason: New leader 9e03f1a6ccf3462dbad7e5ea6892da94. Latest consensus state: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.392720 32261 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:30.393743 32264 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: TxnStatusTablet state changed. Reason: New leader 9e03f1a6ccf3462dbad7e5ea6892da94. Latest consensus state: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.393836 32264 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:30.394429 32268 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.394546 32268 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:30.394611 32268 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.394662 32268 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P 9e03f1a6ccf3462dbad7e5ea6892da94: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:30.394733 32289 mvcc.cc:205] Tried to move back new op lower bound from 7316038166077423616 to 7316038166028427264. Current Snapshot: MvccSnapshot[applied={T|T < 7316038166077423616}]
I20260807 22:35:30.395042 32261 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.395120 32261 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P 4e0741339bf644689b165ba0fd83e669: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:30.395386 32264 tablet_replica.cc:439] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } } }
I20260807 22:35:30.395458 32264 tablet_replica.cc:442] T 5744c143f0a04d08b6a3113892298f43 P a238c20aed7f413b9efd60a30b0a7599: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:30.395627 32290 mvcc.cc:205] Tried to move back new op lower bound from 7316038166077423616 to 7316038166028427264. Current Snapshot: MvccSnapshot[applied={T|T < 7316038166077423616}]
I20260807 22:35:30.395948 32295 mvcc.cc:205] Tried to move back new op lower bound from 7316038166077423616 to 7316038166028427264. Current Snapshot: MvccSnapshot[applied={T|T < 7316038166077423616 or (T in {7316038166077423616})}]
I20260807 22:35:30.402899 32250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.400423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51906 (local address 127.28.82.195:38897)
0807 22:35:30.400471 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.400473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.400481 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:30.400544 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.400545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.400553 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.400648 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:30.400815 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.401310 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.402130 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.402351 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.402430 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.402631 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.402670 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.402735 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.402803 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.402823 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:35:30.407984 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.405345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38897 (local address 127.28.82.195:33395)
0807 22:35:30.405401 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.405417 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.405475 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.405614 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.405615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.405617 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:30.405678 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.405679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.406571 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.406572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.407140 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.407154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.407607 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.407608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.407664 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.407678 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.407851 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.407852 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.407886 (+ 34us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.407919 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 22:35:30.407994 32250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.405419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:33395 (local address 127.28.82.195:38897)
0807 22:35:30.405469 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.405471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.405479 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:30.405534 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.405535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.405544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.405596 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:30.405747 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.406434 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.407233 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.407537 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.407607 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.407762 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.407793 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.407844 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.407889 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.407909 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:35:30.410058 31939 raft_consensus.cc:1261] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Refusing update from remote peer 9e03f1a6ccf3462dbad7e5ea6892da94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:30.410190 31742 raft_consensus.cc:1261] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Refusing update from remote peer 9e03f1a6ccf3462dbad7e5ea6892da94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:30.410286 32268 consensus_queue.cc:1035] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:30.410382 32300 consensus_queue.cc:1035] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:30.423615 31517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:30.423724 31517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:30.424451 31517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: Sending DeleteTablet for 3 replicas of tablet 3bb24fd57a9b4059b102772565d9d827
I20260807 22:35:30.424774 31716 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb24fd57a9b4059b102772565d9d827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:30 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:35916
I20260807 22:35:30.424991 31911 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb24fd57a9b4059b102772565d9d827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:30 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40610
I20260807 22:35:30.425022 32313 tablet_replica.cc:330] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: stopping tablet replica
I20260807 22:35:30.424993 31517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:30.425158 31517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:30.425262 32313 raft_consensus.cc:2229] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:30.425390 32314 tablet_replica.cc:330] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: stopping tablet replica
I20260807 22:35:30.425499 32314 raft_consensus.cc:2229] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:30.425561 32314 pending_rounds.cc:62] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Trying to abort 2 pending ops.
I20260807 22:35:30.425590 32314 pending_rounds.cc:68] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316038166158090240 op_type: PARTICIPANT_OP participant_request { tablet_id: "3bb24fd57a9b4059b102772565d9d827" op { type: BEGIN_TXN txn_id: 0 } }
I20260807 22:35:30.425616 32120 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb24fd57a9b4059b102772565d9d827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:30 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:51894
I20260807 22:35:30.425693 32314 pending_rounds.cc:68] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316038165317615616 op_type: NO_OP noop_request { }
I20260807 22:35:30.425518 32313 raft_consensus.cc:2258] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:30.425743 32314 raft_consensus.cc:2866] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260807 22:35:30.425774 32314 raft_consensus.cc:2258] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:30.426436 31911 tablet_service.cc:1467] Processing CreateTablet for tablet 5440f9095e1147f896c2f7272ad8f06b (DEFAULT_TABLE table=test-table [id=d6af312890f14fa8942fed3543ba389a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:30.426453 32120 tablet_service.cc:1467] Processing CreateTablet for tablet 5440f9095e1147f896c2f7272ad8f06b (DEFAULT_TABLE table=test-table [id=d6af312890f14fa8942fed3543ba389a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:30.426538 31716 tablet_service.cc:1467] Processing CreateTablet for tablet 5440f9095e1147f896c2f7272ad8f06b (DEFAULT_TABLE table=test-table [id=d6af312890f14fa8942fed3543ba389a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:30.426585 32120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5440f9095e1147f896c2f7272ad8f06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:30.426586 31911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5440f9095e1147f896c2f7272ad8f06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:30.426653 31716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5440f9095e1147f896c2f7272ad8f06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:30.428040 32259 tablet_bootstrap.cc:492] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: Bootstrap starting.
W20260807 22:35:30.428021 32310 consensus_queue.cc:1159] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e0741339bf644689b165ba0fd83e669" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260807 22:35:30.428128 32256 tablet_bootstrap.cc:492] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: Bootstrap starting.
I20260807 22:35:30.428380 32314 ts_tablet_manager.cc:1905] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:30.428496 32259 tablet_bootstrap.cc:654] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:30.428848 32256 tablet_bootstrap.cc:654] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:30.428977 32315 tablet_replica.cc:330] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: stopping tablet replica
I20260807 22:35:30.429126 32315 raft_consensus.cc:2229] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:30.429193 32255 tablet_bootstrap.cc:492] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: Bootstrap starting.
I20260807 22:35:30.429234 32315 raft_consensus.cc:2258] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:30.429582 32255 tablet_bootstrap.cc:654] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:30.429663 32256 tablet_bootstrap.cc:492] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: No bootstrap required, opened a new log
I20260807 22:35:30.429709 32256 ts_tablet_manager.cc:1397] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 22:35:30.429854 32256 raft_consensus.cc:348] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.429960 32256 raft_consensus.cc:374] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:30.429987 32256 raft_consensus.cc:729] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a238c20aed7f413b9efd60a30b0a7599, State: Initialized, Role: FOLLOWER
I20260807 22:35:30.430035 32256 consensus_queue.cc:260] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.430140 32256 ts_tablet_manager.cc:1428] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:30.430151 32314 ts_tablet_manager.cc:1918] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:30.430599 32314 log.cc:1196] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/3bb24fd57a9b4059b102772565d9d827
I20260807 22:35:30.430842 32313 ts_tablet_manager.cc:1905] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:30.430948 32314 ts_tablet_manager.cc:1939] T 3bb24fd57a9b4059b102772565d9d827 P 4e0741339bf644689b165ba0fd83e669: Deleting consensus metadata
I20260807 22:35:30.431309 32315 ts_tablet_manager.cc:1905] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:30.431362 31502 catalog_manager.cc:4841] TS 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977): tablet 3bb24fd57a9b4059b102772565d9d827 (table test-table [id=2c1ebc57eac54986a492da19f43ffb36]) successfully deleted
I20260807 22:35:30.432817 32313 ts_tablet_manager.cc:1918] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:30.432907 32313 log.cc:1196] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/3bb24fd57a9b4059b102772565d9d827
I20260807 22:35:30.432976 32315 ts_tablet_manager.cc:1918] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:30.433045 32315 log.cc:1196] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/3bb24fd57a9b4059b102772565d9d827
I20260807 22:35:30.433198 32313 ts_tablet_manager.cc:1939] T 3bb24fd57a9b4059b102772565d9d827 P a238c20aed7f413b9efd60a30b0a7599: Deleting consensus metadata
I20260807 22:35:30.433321 32315 ts_tablet_manager.cc:1939] T 3bb24fd57a9b4059b102772565d9d827 P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting consensus metadata
I20260807 22:35:30.433641 31498 catalog_manager.cc:4841] TS a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565): tablet 3bb24fd57a9b4059b102772565d9d827 (table test-table [id=2c1ebc57eac54986a492da19f43ffb36]) successfully deleted
I20260807 22:35:30.433661 31500 catalog_manager.cc:4841] TS 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897): tablet 3bb24fd57a9b4059b102772565d9d827 (table test-table [id=2c1ebc57eac54986a492da19f43ffb36]) successfully deleted
I20260807 22:35:30.433952 32255 tablet_bootstrap.cc:492] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: No bootstrap required, opened a new log
I20260807 22:35:30.434000 32255 ts_tablet_manager.cc:1397] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 22:35:30.434139 32255 raft_consensus.cc:348] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.434222 32255 raft_consensus.cc:374] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:30.434250 32255 raft_consensus.cc:729] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0741339bf644689b165ba0fd83e669, State: Initialized, Role: FOLLOWER
I20260807 22:35:30.434297 32255 consensus_queue.cc:260] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.434388 32255 ts_tablet_manager.cc:1428] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:30.434849 32259 tablet_bootstrap.cc:492] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: No bootstrap required, opened a new log
I20260807 22:35:30.434914 32259 ts_tablet_manager.cc:1397] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260807 22:35:30.435048 32259 raft_consensus.cc:348] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.435122 32259 raft_consensus.cc:374] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:30.435155 32259 raft_consensus.cc:729] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e03f1a6ccf3462dbad7e5ea6892da94, State: Initialized, Role: FOLLOWER
I20260807 22:35:30.435204 32259 consensus_queue.cc:260] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.435302 32259 ts_tablet_manager.cc:1428] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:30.461375 32264 raft_consensus.cc:482] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:30.461490 32264 raft_consensus.cc:504] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.461826 32264 leader_election.cc:288] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977), 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897)
I20260807 22:35:30.465378 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.462083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:52299 (local address 127.28.82.194:42977)
0807 22:35:30.462133 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.462135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.462279 (+ 144us) server_negotiation.cc:407] Connection header received
0807 22:35:30.462377 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.462378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.462389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.462449 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:30.462639 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.463347 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.464639 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.464998 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.465090 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.465100 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.465260 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.465306 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:30.465395 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.461989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:42977 (local address 127.28.82.193:52299)
0807 22:35:30.462225 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.462234 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.462307 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.462455 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.462455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.462457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:30.462558 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.462561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.463434 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.463435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.464518 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.464520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.465091 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.465092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.465191 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.465196 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.465253 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.465300 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":95,"threads_started":1}
I20260807 22:35:30.465462 32250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.462196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:59187 (local address 127.28.82.195:38897)
0807 22:35:30.462244 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.462246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.462256 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:30.462282 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.462283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.462293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.462372 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:30.462689 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.463138 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.464564 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.465106 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.465189 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.465201 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.465353 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.465392 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 22:35:30.465723 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5440f9095e1147f896c2f7272ad8f06b" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669" is_pre_election: true
I20260807 22:35:30.465819 31939 raft_consensus.cc:2454] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 0.
I20260807 22:35:30.465909 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.461984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38897 (local address 127.28.82.193:59187)
0807 22:35:30.462137 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.462148 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.462202 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.462509 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.462510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.462512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:30.462568 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.462569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.463235 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.463236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.464464 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.464467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.465187 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.465188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.465262 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.465265 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.465722 (+ 457us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.465768 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":57,"threads_started":1}
I20260807 22:35:30.466015 31648 leader_election.cc:302] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0741339bf644689b165ba0fd83e669, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:30.466161 32264 raft_consensus.cc:2781] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:30.466197 32264 raft_consensus.cc:482] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:30.466216 32264 raft_consensus.cc:3037] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.466241 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5440f9095e1147f896c2f7272ad8f06b" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" is_pre_election: true
I20260807 22:35:30.466394 32157 raft_consensus.cc:2454] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 0.
I20260807 22:35:30.467168 32264 raft_consensus.cc:504] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.467334 32264 leader_election.cc:288] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 election: Requested vote from peers 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977), 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897)
I20260807 22:35:30.467490 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5440f9095e1147f896c2f7272ad8f06b" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94"
I20260807 22:35:30.467490 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5440f9095e1147f896c2f7272ad8f06b" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669"
I20260807 22:35:30.467561 32157 raft_consensus.cc:3037] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.467561 31939 raft_consensus.cc:3037] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:30.468324 32157 raft_consensus.cc:2454] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 1.
I20260807 22:35:30.468324 31939 raft_consensus.cc:2454] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 1.
I20260807 22:35:30.468508 31647 leader_election.cc:302] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e03f1a6ccf3462dbad7e5ea6892da94, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:30.468613 32264 raft_consensus.cc:2781] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:30.468816 32264 raft_consensus.cc:686] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 1 LEADER]: Becoming Leader. State: Replica: a238c20aed7f413b9efd60a30b0a7599, State: Running, Role: LEADER
I20260807 22:35:30.468901 32264 consensus_queue.cc:237] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:30.469609 31517 catalog_manager.cc:5495] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "a238c20aed7f413b9efd60a30b0a7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } health_report { overall_health: UNKNOWN } } }
I20260807 22:35:30.482910 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.480660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.28.82.193:44565)
0807 22:35:30.480705 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.480707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.480714 (+ 7us) server_negotiation.cc:407] Connection header received
0807 22:35:30.480746 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.480747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.480766 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.480845 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:30.480954 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.481426 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.482124 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.482326 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.482403 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.482574 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.482682 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.482758 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.482814 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.482833 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:30.488027 32328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.485323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38897 (local address 127.28.82.193:36297)
0807 22:35:30.485474 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.485485 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.485541 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.485661 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.485662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.485663 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:30.485722 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.485723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.486582 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.486582 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.487172 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.487174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.487610 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.487611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.487672 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.487687 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.487883 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.487884 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.487923 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.487956 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260807 22:35:30.488021 32250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.485395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:36297 (local address 127.28.82.195:38897)
0807 22:35:30.485442 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.485444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.485549 (+ 105us) server_negotiation.cc:407] Connection header received
0807 22:35:30.485593 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.485594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.485604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.485661 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:30.485765 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.486509 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.487252 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.487528 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.487607 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.487770 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.487801 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.487873 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.487929 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.487948 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 22:35:30.490089 32157 raft_consensus.cc:1261] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Refusing update from remote peer a238c20aed7f413b9efd60a30b0a7599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:30.490438 31939 raft_consensus.cc:1261] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Refusing update from remote peer a238c20aed7f413b9efd60a30b0a7599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:30.490831 32324 consensus_queue.cc:1035] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:30.490965 32264 consensus_queue.cc:1035] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:30.498986 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.496519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:44565 (local address 127.28.82.195:42143)
0807 22:35:30.496561 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:30.496571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:30.496607 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:30.496788 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:30.496789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:30.496790 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:30.496834 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.496835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.497612 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.497613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.498150 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:30.498151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.498541 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.498542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:30.498602 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.498606 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.498817 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.498818 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 22:35:30.498871 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 22:35:30.498915 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 22:35:30.498986 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:30.496575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:42143 (local address 127.28.82.193:44565)
0807 22:35:30.496671 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:30.496673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:30.496681 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:30.496702 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:30.496702 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:30.496711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:30.496783 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:30.496885 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.497539 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.498232 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:30.498433 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:30.498541 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:30.498678 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:30.498734 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:30.498811 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:30.498892 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 22:35:30.498913 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 22:35:30.499317 32295 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 22:35:32.022395 31518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:32.022527 31518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:32.023386 31518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: Sending DeleteTablet for 3 replicas of tablet 5440f9095e1147f896c2f7272ad8f06b
I20260807 22:35:32.023717 31911 tablet_service.cc:1514] Processing DeleteTablet for tablet 5440f9095e1147f896c2f7272ad8f06b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:32 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40610
I20260807 22:35:32.023744 32120 tablet_service.cc:1514] Processing DeleteTablet for tablet 5440f9095e1147f896c2f7272ad8f06b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:32 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:51894
I20260807 22:35:32.023949 32365 tablet_replica.cc:330] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: stopping tablet replica
I20260807 22:35:32.024017 32365 raft_consensus.cc:2229] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:32.024066 32365 raft_consensus.cc:2258] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:32.026575 31716 tablet_service.cc:1514] Processing DeleteTablet for tablet 5440f9095e1147f896c2f7272ad8f06b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:32 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:35916
I20260807 22:35:32.026790 32368 tablet_replica.cc:330] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: stopping tablet replica
I20260807 22:35:32.026906 32368 raft_consensus.cc:2229] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:32.027052 32368 raft_consensus.cc:2258] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:32.027359 32368 ts_tablet_manager.cc:1905] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:32.027763 32365 ts_tablet_manager.cc:1905] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:32.029659 32368 ts_tablet_manager.cc:1918] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:35:32.029731 32368 log.cc:1196] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/5440f9095e1147f896c2f7272ad8f06b
I20260807 22:35:32.029870 32365 ts_tablet_manager.cc:1918] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:35:32.029937 32365 log.cc:1196] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/5440f9095e1147f896c2f7272ad8f06b
I20260807 22:35:32.030006 32368 ts_tablet_manager.cc:1939] T 5440f9095e1147f896c2f7272ad8f06b P a238c20aed7f413b9efd60a30b0a7599: Deleting consensus metadata
I20260807 22:35:32.030176 32365 ts_tablet_manager.cc:1939] T 5440f9095e1147f896c2f7272ad8f06b P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting consensus metadata
I20260807 22:35:32.030324 31498 catalog_manager.cc:4841] TS a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565): tablet 5440f9095e1147f896c2f7272ad8f06b (table test-table [id=d6af312890f14fa8942fed3543ba389a]) successfully deleted
I20260807 22:35:32.030541 32364 tablet_replica.cc:330] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: stopping tablet replica
I20260807 22:35:32.030638 32364 raft_consensus.cc:2229] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:32.030678 31500 catalog_manager.cc:4841] TS 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897): tablet 5440f9095e1147f896c2f7272ad8f06b (table test-table [id=d6af312890f14fa8942fed3543ba389a]) successfully deleted
I20260807 22:35:32.030720 32364 raft_consensus.cc:2258] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:32.030946 32364 ts_tablet_manager.cc:1905] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:32.032233 32364 ts_tablet_manager.cc:1918] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:35:32.032308 32364 log.cc:1196] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/5440f9095e1147f896c2f7272ad8f06b
I20260807 22:35:32.032516 32364 ts_tablet_manager.cc:1939] T 5440f9095e1147f896c2f7272ad8f06b P 4e0741339bf644689b165ba0fd83e669: Deleting consensus metadata
I20260807 22:35:32.032835 31502 catalog_manager.cc:4841] TS 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977): tablet 5440f9095e1147f896c2f7272ad8f06b (table test-table [id=d6af312890f14fa8942fed3543ba389a]) successfully deleted
Aug 07 22:35:32 dist-test-slave-7kzw krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142132, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:35:32 dist-test-slave-7kzw krb5kdc[31450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142132, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:32.053884 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:32.044040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.28.82.254:44091)
0807 22:35:32.044192 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:32.044194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:32.044204 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:32.044247 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:32.044247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:32.044258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:32.044318 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:32.044932 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:32.045488 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:32.046239 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:32.046430 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:32.046555 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:32.049519 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:32.049521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:32.049521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:32.049530 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:32.051762 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:32.052211 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:32.052213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:32.052213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:32.052296 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:32.052724 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:32.052725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:32.052726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:32.052983 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:32.053353 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:32.053479 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 22:35:32.053736 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260807 22:35:32.060068 31518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:32.060350 31518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:32.061987 32120 tablet_service.cc:1467] Processing CreateTablet for tablet 5302e98b974b498b99285b9228c8fc1a (DEFAULT_TABLE table=test-table [id=b9579abc697a40e69e4201154536adf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:32.062007 31911 tablet_service.cc:1467] Processing CreateTablet for tablet 5302e98b974b498b99285b9228c8fc1a (DEFAULT_TABLE table=test-table [id=b9579abc697a40e69e4201154536adf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:32.062155 31911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5302e98b974b498b99285b9228c8fc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:32.062143 32120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5302e98b974b498b99285b9228c8fc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:32.062147 31716 tablet_service.cc:1467] Processing CreateTablet for tablet 5302e98b974b498b99285b9228c8fc1a (DEFAULT_TABLE table=test-table [id=b9579abc697a40e69e4201154536adf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:32.062265 31716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5302e98b974b498b99285b9228c8fc1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:32.063894 32382 tablet_bootstrap.cc:492] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: Bootstrap starting.
I20260807 22:35:32.063926 32381 tablet_bootstrap.cc:492] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: Bootstrap starting.
I20260807 22:35:32.064371 32381 tablet_bootstrap.cc:654] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:32.064384 32382 tablet_bootstrap.cc:654] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:32.064821 32379 tablet_bootstrap.cc:492] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: Bootstrap starting.
I20260807 22:35:32.065198 32382 tablet_bootstrap.cc:492] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: No bootstrap required, opened a new log
I20260807 22:35:32.065238 32379 tablet_bootstrap.cc:654] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:32.065265 32382 ts_tablet_manager.cc:1397] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:32.065487 32382 raft_consensus.cc:348] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.065675 32382 raft_consensus.cc:374] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:32.065768 32382 raft_consensus.cc:729] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0741339bf644689b165ba0fd83e669, State: Initialized, Role: FOLLOWER
I20260807 22:35:32.065768 32381 tablet_bootstrap.cc:492] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: No bootstrap required, opened a new log
I20260807 22:35:32.065825 32381 ts_tablet_manager.cc:1397] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:32.065860 32382 consensus_queue.cc:260] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.066080 32382 ts_tablet_manager.cc:1428] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:32.066071 32381 raft_consensus.cc:348] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.066154 32381 raft_consensus.cc:374] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:32.066176 32381 raft_consensus.cc:729] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a238c20aed7f413b9efd60a30b0a7599, State: Initialized, Role: FOLLOWER
I20260807 22:35:32.066344 32379 tablet_bootstrap.cc:492] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: No bootstrap required, opened a new log
I20260807 22:35:32.066429 32379 ts_tablet_manager.cc:1397] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:32.066553 32381 consensus_queue.cc:260] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.066744 32379 raft_consensus.cc:348] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.067003 32381 ts_tablet_manager.cc:1428] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:35:32.067122 32379 raft_consensus.cc:374] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:32.067190 32379 raft_consensus.cc:729] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e03f1a6ccf3462dbad7e5ea6892da94, State: Initialized, Role: FOLLOWER
I20260807 22:35:32.067291 32379 consensus_queue.cc:260] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.067487 32379 ts_tablet_manager.cc:1428] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 22:35:32.122008 32352 raft_consensus.cc:482] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:32.122129 32352 raft_consensus.cc:504] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.122345 32352 leader_election.cc:288] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977), 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897)
I20260807 22:35:32.122570 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5302e98b974b498b99285b9228c8fc1a" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669" is_pre_election: true
I20260807 22:35:32.122707 31939 raft_consensus.cc:2454] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 0.
I20260807 22:35:32.122651 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5302e98b974b498b99285b9228c8fc1a" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" is_pre_election: true
I20260807 22:35:32.122777 32157 raft_consensus.cc:2454] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 0.
I20260807 22:35:32.122898 31648 leader_election.cc:302] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0741339bf644689b165ba0fd83e669, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:32.123020 32352 raft_consensus.cc:2781] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:32.123085 32352 raft_consensus.cc:482] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:32.123113 32352 raft_consensus.cc:3037] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:32.123894 32352 raft_consensus.cc:504] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.124044 32352 leader_election.cc:288] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 election: Requested vote from peers 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977), 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897)
I20260807 22:35:32.124209 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5302e98b974b498b99285b9228c8fc1a" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669"
I20260807 22:35:32.124207 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5302e98b974b498b99285b9228c8fc1a" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94"
I20260807 22:35:32.124296 31939 raft_consensus.cc:3037] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:32.124294 32157 raft_consensus.cc:3037] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:32.124923 32157 raft_consensus.cc:2454] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 1.
I20260807 22:35:32.124938 31939 raft_consensus.cc:2454] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 1.
I20260807 22:35:32.125097 31647 leader_election.cc:302] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e03f1a6ccf3462dbad7e5ea6892da94, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:32.125227 32352 raft_consensus.cc:2781] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:32.125308 32352 raft_consensus.cc:686] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 1 LEADER]: Becoming Leader. State: Replica: a238c20aed7f413b9efd60a30b0a7599, State: Running, Role: LEADER
I20260807 22:35:32.125362 32352 consensus_queue.cc:237] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.125996 31516 catalog_manager.cc:5495] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "a238c20aed7f413b9efd60a30b0a7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } health_report { overall_health: UNKNOWN } } }
I20260807 22:35:32.151467 32392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:32.141582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35922 (local address 127.28.82.193:44565)
0807 22:35:32.141747 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:32.141749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:32.141759 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:32.141788 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:32.141789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:32.141799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:32.141866 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:32.141983 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:32.142635 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:32.150345 (+ 7710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:32.150815 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:32.150918 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:32.151113 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:32.151176 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:32.151240 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:32.151311 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 22:35:32.151345 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":114,"threads_started":1}
I20260807 22:35:32.153956 32157 raft_consensus.cc:1261] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Refusing update from remote peer a238c20aed7f413b9efd60a30b0a7599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:32.153988 31939 raft_consensus.cc:1261] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Refusing update from remote peer a238c20aed7f413b9efd60a30b0a7599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:32.154245 32332 consensus_queue.cc:1035] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:32.154356 32352 consensus_queue.cc:1035] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:32.164070 31518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:32.164155 31518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:32.164812 31518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: Sending DeleteTablet for 3 replicas of tablet 5302e98b974b498b99285b9228c8fc1a
I20260807 22:35:32.165082 31716 tablet_service.cc:1514] Processing DeleteTablet for tablet 5302e98b974b498b99285b9228c8fc1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:32 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:35916
I20260807 22:35:32.165082 31911 tablet_service.cc:1514] Processing DeleteTablet for tablet 5302e98b974b498b99285b9228c8fc1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:32 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40610
I20260807 22:35:32.165186 32368 tablet_replica.cc:330] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: stopping tablet replica
I20260807 22:35:32.165211 32364 tablet_replica.cc:330] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: stopping tablet replica
I20260807 22:35:32.165239 32368 raft_consensus.cc:2229] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:32.165269 32364 raft_consensus.cc:2229] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:32.165292 32368 raft_consensus.cc:2258] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:32.165310 32364 raft_consensus.cc:2258] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:32.165293 32120 tablet_service.cc:1514] Processing DeleteTablet for tablet 5302e98b974b498b99285b9228c8fc1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:32 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:51894
I20260807 22:35:32.165446 32365 tablet_replica.cc:330] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: stopping tablet replica
I20260807 22:35:32.165503 32365 raft_consensus.cc:2229] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:32.165544 32365 raft_consensus.cc:2258] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:32.165633 32364 ts_tablet_manager.cc:1905] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:32.165678 32368 ts_tablet_manager.cc:1905] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:32.165673 31518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:32.165820 31518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:32.166136 32365 ts_tablet_manager.cc:1905] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:32.167294 32364 ts_tablet_manager.cc:1918] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:32.167364 32364 log.cc:1196] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/5302e98b974b498b99285b9228c8fc1a
I20260807 22:35:32.167371 32368 ts_tablet_manager.cc:1918] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:32.167433 32368 log.cc:1196] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/5302e98b974b498b99285b9228c8fc1a
I20260807 22:35:32.167605 32364 ts_tablet_manager.cc:1939] T 5302e98b974b498b99285b9228c8fc1a P 4e0741339bf644689b165ba0fd83e669: Deleting consensus metadata
I20260807 22:35:32.167649 32368 ts_tablet_manager.cc:1939] T 5302e98b974b498b99285b9228c8fc1a P a238c20aed7f413b9efd60a30b0a7599: Deleting consensus metadata
I20260807 22:35:32.167906 32365 ts_tablet_manager.cc:1918] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:32.167912 31498 catalog_manager.cc:4841] TS a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565): tablet 5302e98b974b498b99285b9228c8fc1a (table test-table [id=b9579abc697a40e69e4201154536adf6]) successfully deleted
I20260807 22:35:32.167968 32365 log.cc:1196] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/5302e98b974b498b99285b9228c8fc1a
I20260807 22:35:32.168190 32365 ts_tablet_manager.cc:1939] T 5302e98b974b498b99285b9228c8fc1a P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting consensus metadata
I20260807 22:35:32.168468 31500 catalog_manager.cc:4841] TS 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897): tablet 5302e98b974b498b99285b9228c8fc1a (table test-table [id=b9579abc697a40e69e4201154536adf6]) successfully deleted
I20260807 22:35:32.168483 31502 catalog_manager.cc:4841] TS 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977): tablet 5302e98b974b498b99285b9228c8fc1a (table test-table [id=b9579abc697a40e69e4201154536adf6]) successfully deleted
I20260807 22:35:32.169183 31716 tablet_service.cc:1467] Processing CreateTablet for tablet f510603aa01a4fda87cf6eaac85b440b (DEFAULT_TABLE table=test-table [id=58fd48d8e14540768e8191d82c7e1755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:32.169203 31911 tablet_service.cc:1467] Processing CreateTablet for tablet f510603aa01a4fda87cf6eaac85b440b (DEFAULT_TABLE table=test-table [id=58fd48d8e14540768e8191d82c7e1755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:32.169313 31911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f510603aa01a4fda87cf6eaac85b440b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:32.169317 31716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f510603aa01a4fda87cf6eaac85b440b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:32.170140 32381 tablet_bootstrap.cc:492] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599: Bootstrap starting.
I20260807 22:35:32.170186 32382 tablet_bootstrap.cc:492] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: Bootstrap starting.
I20260807 22:35:32.170298 32120 tablet_service.cc:1467] Processing CreateTablet for tablet f510603aa01a4fda87cf6eaac85b440b (DEFAULT_TABLE table=test-table [id=58fd48d8e14540768e8191d82c7e1755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:32.170403 32120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f510603aa01a4fda87cf6eaac85b440b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:32.170519 32381 tablet_bootstrap.cc:654] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:32.170846 32382 tablet_bootstrap.cc:654] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:32.171449 32382 tablet_bootstrap.cc:492] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: No bootstrap required, opened a new log
I20260807 22:35:32.171515 32382 ts_tablet_manager.cc:1397] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:32.171665 32382 raft_consensus.cc:348] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.171739 32382 raft_consensus.cc:374] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:32.171728 32379 tablet_bootstrap.cc:492] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: Bootstrap starting.
I20260807 22:35:32.171768 32382 raft_consensus.cc:729] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0741339bf644689b165ba0fd83e669, State: Initialized, Role: FOLLOWER
I20260807 22:35:32.171814 32382 consensus_queue.cc:260] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.171911 32382 ts_tablet_manager.cc:1428] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:32.172029 32379 tablet_bootstrap.cc:654] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:32.172497 32381 tablet_bootstrap.cc:492] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599: No bootstrap required, opened a new log
I20260807 22:35:32.172555 32381 ts_tablet_manager.cc:1397] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:32.172569 32379 tablet_bootstrap.cc:492] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: No bootstrap required, opened a new log
I20260807 22:35:32.172612 32379 ts_tablet_manager.cc:1397] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:32.172720 32381 raft_consensus.cc:348] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.172720 32379 raft_consensus.cc:348] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.172791 32379 raft_consensus.cc:374] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:32.172791 32381 raft_consensus.cc:374] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:32.172813 32379 raft_consensus.cc:729] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e03f1a6ccf3462dbad7e5ea6892da94, State: Initialized, Role: FOLLOWER
I20260807 22:35:32.172813 32381 raft_consensus.cc:729] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a238c20aed7f413b9efd60a30b0a7599, State: Initialized, Role: FOLLOWER
I20260807 22:35:32.172854 32379 consensus_queue.cc:260] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.172856 32381 consensus_queue.cc:260] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.172952 32381 ts_tablet_manager.cc:1428] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:32.172948 32379 ts_tablet_manager.cc:1428] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:32.420038 32332 raft_consensus.cc:482] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:32.420167 32332 raft_consensus.cc:504] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.420369 32332 leader_election.cc:288] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977), 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897)
I20260807 22:35:32.420647 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f510603aa01a4fda87cf6eaac85b440b" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" is_pre_election: true
I20260807 22:35:32.420779 32157 raft_consensus.cc:2454] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 0.
I20260807 22:35:32.420702 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f510603aa01a4fda87cf6eaac85b440b" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669" is_pre_election: true
I20260807 22:35:32.420838 31939 raft_consensus.cc:2454] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 0.
I20260807 22:35:32.420957 31647 leader_election.cc:302] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e03f1a6ccf3462dbad7e5ea6892da94, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:32.421075 32332 raft_consensus.cc:2781] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:32.421118 32332 raft_consensus.cc:482] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:32.421144 32332 raft_consensus.cc:3037] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:32.421934 32332 raft_consensus.cc:504] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.422108 32332 leader_election.cc:288] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 election: Requested vote from peers 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977), 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897)
I20260807 22:35:32.422282 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f510603aa01a4fda87cf6eaac85b440b" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0741339bf644689b165ba0fd83e669"
I20260807 22:35:32.422361 31939 raft_consensus.cc:3037] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:32.422345 32157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f510603aa01a4fda87cf6eaac85b440b" candidate_uuid: "a238c20aed7f413b9efd60a30b0a7599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94"
I20260807 22:35:32.422449 32157 raft_consensus.cc:3037] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:32.423130 31939 raft_consensus.cc:2454] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 1.
I20260807 22:35:32.423308 32157 raft_consensus.cc:2454] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a238c20aed7f413b9efd60a30b0a7599 in term 1.
I20260807 22:35:32.423343 31648 leader_election.cc:302] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0741339bf644689b165ba0fd83e669, a238c20aed7f413b9efd60a30b0a7599; no voters:
I20260807 22:35:32.423470 32332 raft_consensus.cc:2781] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:32.423537 32332 raft_consensus.cc:686] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [term 1 LEADER]: Becoming Leader. State: Replica: a238c20aed7f413b9efd60a30b0a7599, State: Running, Role: LEADER
I20260807 22:35:32.423585 32332 consensus_queue.cc:237] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } }
I20260807 22:35:32.424191 31516 catalog_manager.cc:5495] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 reported cstate change: term changed from 0 to 1, leader changed from <none> to a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "a238c20aed7f413b9efd60a30b0a7599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a238c20aed7f413b9efd60a30b0a7599" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 44565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 } health_report { overall_health: UNKNOWN } } }
I20260807 22:35:32.457599 31939 raft_consensus.cc:1261] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Refusing update from remote peer a238c20aed7f413b9efd60a30b0a7599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:32.457599 32157 raft_consensus.cc:1261] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Refusing update from remote peer a238c20aed7f413b9efd60a30b0a7599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:32.457857 32397 consensus_queue.cc:1035] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0741339bf644689b165ba0fd83e669" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:32.457989 32332 consensus_queue.cc:1035] T f510603aa01a4fda87cf6eaac85b440b P a238c20aed7f413b9efd60a30b0a7599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e03f1a6ccf3462dbad7e5ea6892da94" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:35:32.462162 32342 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 22:35:33.979905 31516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:33.980022 31516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:33.980875 31516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a08dbc76e9e4b79a3628c3611928625: Sending DeleteTablet for 3 replicas of tablet f510603aa01a4fda87cf6eaac85b440b
I20260807 22:35:33.981143 31716 tablet_service.cc:1514] Processing DeleteTablet for tablet f510603aa01a4fda87cf6eaac85b440b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:33 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:35916
I20260807 22:35:33.981146 31911 tablet_service.cc:1514] Processing DeleteTablet for tablet f510603aa01a4fda87cf6eaac85b440b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:33 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40610
I20260807 22:35:33.981271 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 31624
I20260807 22:35:33.981597 32122 tablet_service.cc:1514] Processing DeleteTablet for tablet f510603aa01a4fda87cf6eaac85b440b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:33 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:51894
I20260807 22:35:33.986438 32451 tablet_replica.cc:330] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: stopping tablet replica
I20260807 22:35:33.986573 32451 raft_consensus.cc:2229] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:33.986652 32451 raft_consensus.cc:2258] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:33.987860 32451 ts_tablet_manager.cc:1905] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:33.989444 32448 tablet_replica.cc:330] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: stopping tablet replica
I20260807 22:35:33.989528 32448 raft_consensus.cc:2229] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:33.989542 32451 ts_tablet_manager.cc:1918] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:35:33.989599 32448 raft_consensus.cc:2258] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:33.989631 32451 log.cc:1196] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/f510603aa01a4fda87cf6eaac85b440b
I20260807 22:35:33.989873 32451 ts_tablet_manager.cc:1939] T f510603aa01a4fda87cf6eaac85b440b P 9e03f1a6ccf3462dbad7e5ea6892da94: Deleting consensus metadata
I20260807 22:35:33.990228 31500 catalog_manager.cc:4841] TS 9e03f1a6ccf3462dbad7e5ea6892da94 (127.28.82.195:38897): tablet f510603aa01a4fda87cf6eaac85b440b (table test-table [id=58fd48d8e14540768e8191d82c7e1755]) successfully deleted
I20260807 22:35:33.990319 32448 ts_tablet_manager.cc:1905] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:33.991529 32448 ts_tablet_manager.cc:1918] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 22:35:33.991603 32448 log.cc:1196] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/f510603aa01a4fda87cf6eaac85b440b
I20260807 22:35:33.991822 32448 ts_tablet_manager.cc:1939] T f510603aa01a4fda87cf6eaac85b440b P 4e0741339bf644689b165ba0fd83e669: Deleting consensus metadata
I20260807 22:35:33.992151 31502 catalog_manager.cc:4841] TS 4e0741339bf644689b165ba0fd83e669 (127.28.82.194:42977): tablet f510603aa01a4fda87cf6eaac85b440b (table test-table [id=58fd48d8e14540768e8191d82c7e1755]) successfully deleted
W20260807 22:35:33.995669 31498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:44565 (error 108)
I20260807 22:35:33.995975 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 31819
I20260807 22:35:33.998713 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:33.996041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:44565 (local address 127.0.0.1:39208)
0807 22:35:33.998622 (+ 2581us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:33.998674 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:44565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2555,"thread_start_us":69,"threads_started":1}
W20260807 22:35:34.000185 31498 catalog_manager.cc:4584] TS a238c20aed7f413b9efd60a30b0a7599 (127.28.82.193:44565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f510603aa01a4fda87cf6eaac85b440b: Network error: Client connection negotiation failed: client connection to 127.28.82.193:44565: connect: Connection refused (error 111)
I20260807 22:35:34.009213 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 32025
I20260807 22:35:34.022629 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 31475
2026-08-07T22:35:34Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6087 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:34 dist-test-slave-7kzw krb5kdc[32457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:34 dist-test-slave-7kzw krb5kdc[32457](info): set up 2 sockets
Aug 07 22:35:34 dist-test-slave-7kzw krb5kdc[32457](info): commencing operation
krb5kdc: starting...
W20260807 22:35:35.316443 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260807 22:35:35.605513 29003 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.289s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260807 22:35:35.841372 29003 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.236s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260807 22:35:36.169262 29003 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.328s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/test-user.keytab.
W20260807 22:35:36.357393 29003 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.188s user 0.000s sys 0.002s
Aug 07 22:35:36 dist-test-slave-7kzw krb5kdc[32457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:36.403298 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39847
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:36.486208 32474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:36.486416 32474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:36.486439 32474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:36.487634 32474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:36.487679 32474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:36.487694 32474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:36.487706 32474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:36.487720 32474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:36.489126 32474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:36.489322 32474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:36.489494 32474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:36.494738 32489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:36.494836 32474 server_base.cc:1034] running on GCE node
W20260807 22:35:36.494742 32481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:36.494755 32483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:36.495309 32474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:36.495560 32474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:36.496726 32474 hybrid_clock.cc:648] HybridClock initialized: now 1786142136496711 us; error 31 us; skew 500 ppm
Aug 07 22:35:36 dist-test-slave-7kzw krb5kdc[32457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142136, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:36.503203 32474 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:36.503702 32474 webserver.cc:460] Webserver started at http://127.28.82.254:43767/ using document root <none> and password file <none>
I20260807 22:35:36.503957 32474 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:36.504004 32474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:36.504184 32474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:36.505254 32474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "ba45369b3ce4415b97a5598cb6f6b92a"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a708a8874bef7de2dde19b7f580c140c"
server_key_iv: "78131c4323e3e06620103c3580946830"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.505636 32474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "ba45369b3ce4415b97a5598cb6f6b92a"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a708a8874bef7de2dde19b7f580c140c"
server_key_iv: "78131c4323e3e06620103c3580946830"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.507078 32474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:36.507948 32496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:36.508201 32474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:36.508311 32474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "ba45369b3ce4415b97a5598cb6f6b92a"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a708a8874bef7de2dde19b7f580c140c"
server_key_iv: "78131c4323e3e06620103c3580946830"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.508399 32474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:36.524147 32474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:36.526068 32474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:36.526237 32474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:36.530638 32474 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:39847
I20260807 22:35:36.530712 32577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:39847 every 8 connection(s)
I20260807 22:35:36.531062 32474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:36.531966 32578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:36.534293 32578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a: Bootstrap starting.
I20260807 22:35:36.534945 32578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:36.535249 32578 log.cc:824] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:36.535989 32578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a: No bootstrap required, opened a new log
I20260807 22:35:36.537670 32578 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba45369b3ce4415b97a5598cb6f6b92a" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39847 } }
I20260807 22:35:36.537804 32578 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:36.537828 32578 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba45369b3ce4415b97a5598cb6f6b92a, State: Initialized, Role: FOLLOWER
I20260807 22:35:36.537997 32578 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba45369b3ce4415b97a5598cb6f6b92a" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39847 } }
I20260807 22:35:36.538070 32578 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:36.538112 32578 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:36.538166 32578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:36.538700 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 32474
I20260807 22:35:36.538779 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:36.538726 32578 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba45369b3ce4415b97a5598cb6f6b92a" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39847 } }
I20260807 22:35:36.538857 32578 leader_election.cc:302] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba45369b3ce4415b97a5598cb6f6b92a; no voters:
I20260807 22:35:36.538975 29003 external_mini_cluster.cc:1428] Setting key 8d2282ad61c557c8f7cbb15572263e26
I20260807 22:35:36.539053 32578 leader_election.cc:288] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:36.539115 32582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:36.539248 32582 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [term 1 LEADER]: Becoming Leader. State: Replica: ba45369b3ce4415b97a5598cb6f6b92a, State: Running, Role: LEADER
I20260807 22:35:36.539330 32578 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:36.539389 32582 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba45369b3ce4415b97a5598cb6f6b92a" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39847 } }
I20260807 22:35:36.539712 32582 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba45369b3ce4415b97a5598cb6f6b92a. Latest consensus state: current_term: 1 leader_uuid: "ba45369b3ce4415b97a5598cb6f6b92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba45369b3ce4415b97a5598cb6f6b92a" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39847 } } }
I20260807 22:35:36.539731 32584 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba45369b3ce4415b97a5598cb6f6b92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba45369b3ce4415b97a5598cb6f6b92a" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39847 } } }
I20260807 22:35:36.539892 32582 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:36.540186 32584 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:36.540271 32595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:36.540864 32595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:36.543708 32595 catalog_manager.cc:1347] Generated new cluster ID: 76a84e425d7d4180bac7c22b53427e35
I20260807 22:35:36.543766 32595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:35:36 dist-test-slave-7kzw krb5kdc[32457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:36.550761 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:36.540077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.28.82.254:39847)
0807 22:35:36.540887 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:36.540896 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:36.540932 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:35:36.541053 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:36.541062 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:36.541121 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:36.541273 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:36.542072 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.543082 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.544021 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.544428 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.544545 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:36.546885 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:36.546889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:36.546889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:36.546902 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:36.549177 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:36.549552 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:36.549555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:36.549556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:36.549661 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:36.549881 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:36.549882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:36.549882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:36.550127 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:36.550318 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:36.550383 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 22:35:36.550553 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":74,"threads_started":1}
I20260807 22:35:36.561009 32595 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:36.561520 32595 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:36.565907 32595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba45369b3ce4415b97a5598cb6f6b92a: Generated new TSK 0
I20260807 22:35:36.566084 32595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:36.598178 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39847
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:36.680862 32620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:36.681059 32620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:36.681082 32620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:36.682238 32620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:36.682281 32620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:36.682334 32620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:36.683849 32620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:36.684094 32620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:36.684340 32620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:36.689333 32631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:36.689356 32627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:36.689333 32628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:36.689659 32620 server_base.cc:1034] running on GCE node
I20260807 22:35:36.689904 32620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:36.690171 32620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:36.691323 32620 hybrid_clock.cc:648] HybridClock initialized: now 1786142136691313 us; error 32 us; skew 500 ppm
Aug 07 22:35:36 dist-test-slave-7kzw krb5kdc[32457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142136, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:36.693804 32620 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:36.694296 32620 webserver.cc:460] Webserver started at http://127.28.82.193:41415/ using document root <none> and password file <none>
I20260807 22:35:36.694634 32620 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:36.694710 32620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:36.694854 32620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:36.695912 32620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "1c7b288056f54bfbbac558dcbb551290"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a5ba88af0a2f53f03ed18e9d4a91c6bf"
server_key_iv: "f7b69f99066d3d625f1ab6df653f6c29"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.696280 32620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "1c7b288056f54bfbbac558dcbb551290"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a5ba88af0a2f53f03ed18e9d4a91c6bf"
server_key_iv: "f7b69f99066d3d625f1ab6df653f6c29"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.697634 32620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:36.698539 32638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:36.698776 32620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:36.698900 32620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "1c7b288056f54bfbbac558dcbb551290"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a5ba88af0a2f53f03ed18e9d4a91c6bf"
server_key_iv: "f7b69f99066d3d625f1ab6df653f6c29"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.698987 32620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:36.714046 32620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:36.716542 32620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:36.716672 32620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:36.716893 32620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:36.717181 32620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:36.717216 32620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:36.717239 32620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:36.717253 32620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:36.723555 32620 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:44303
I20260807 22:35:36.723582 332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:44303 every 8 connection(s)
I20260807 22:35:36.723946 32620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:35:36 dist-test-slave-7kzw krb5kdc[32457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142136, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:36.733812 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 32620
I20260807 22:35:36.733901 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:36.724846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:46961 (local address 127.28.82.254:39847)
0807 22:35:36.724924 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:36.724927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:36.725103 (+ 176us) server_negotiation.cc:407] Connection header received
0807 22:35:36.725638 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:36.725640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:36.725658 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:36.725705 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:36.726753 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.727265 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.728289 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.728498 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.728574 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:36.731136 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:36.731138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:36.731138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:36.731146 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:36.732601 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:36.733001 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:36.733002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:36.733003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:36.733057 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:36.733274 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:36.733274 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:36.733275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:36.733377 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:36.733428 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:36.733675 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 22:35:36.733770 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:35:36.733915 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:36.734051 340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:36.724707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39847 (local address 127.28.82.193:46961)
0807 22:35:36.724965 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:36.725019 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:36.725521 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:36.725795 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:36.725801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:36.726375 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:36.726635 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:36.726640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.727379 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.727381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:36.728183 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:36.728185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.728582 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.728583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:36.728735 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:36.729307 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:36.729319 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:36.731044 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:36.732693 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:36.732716 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:36.732718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:36.732933 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:36.733117 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:36.733118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:36.733118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:36.733219 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:36.733432 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:36.733437 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:36.733548 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 22:35:36.733629 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 22:35:36.733779 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":122,"threads_started":1}
I20260807 22:35:36.734122 29003 external_mini_cluster.cc:1428] Setting key 8f90a285200579da14fba4b760bbec95
I20260807 22:35:36.734582 333 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39847
I20260807 22:35:36.734685 333 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:36.734867 333 heartbeater.cc:511] Master 127.28.82.254:39847 requested a full tablet report, sending...
I20260807 22:35:36.735383 32522 ts_manager.cc:194] Registered new tserver with Master: 1c7b288056f54bfbbac558dcbb551290 (127.28.82.193:44303)
I20260807 22:35:36.735987 32522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:46961
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:36.759406 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39847
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:36.843886 344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:36.844110 344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:36.844134 344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:36.845383 344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:36.845429 344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:36.845456 344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:36.846935 344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:36.847131 344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:36.847326 344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:36.852357 350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:36.852344 349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:36.852329 352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:36.852748 344 server_base.cc:1034] running on GCE node
I20260807 22:35:36.852986 344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:36.853250 344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:36.854454 344 hybrid_clock.cc:648] HybridClock initialized: now 1786142136854440 us; error 33 us; skew 500 ppm
Aug 07 22:35:36 dist-test-slave-7kzw krb5kdc[32457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142136, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:36.857389 344 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:36.857894 344 webserver.cc:460] Webserver started at http://127.28.82.194:35349/ using document root <none> and password file <none>
I20260807 22:35:36.858161 344 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:36.858238 344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:36.858397 344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:36.859479 344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "69f0c3bbd7cb485b8496edea2e2d1668"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a8a3ea4ff6ff4788bd81b009b9459e34"
server_key_iv: "820995a4b982e3daf4442d4f4250d160"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.859848 344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "69f0c3bbd7cb485b8496edea2e2d1668"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a8a3ea4ff6ff4788bd81b009b9459e34"
server_key_iv: "820995a4b982e3daf4442d4f4250d160"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.861230 344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:36.861971 363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:36.862198 344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:36.862293 344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "69f0c3bbd7cb485b8496edea2e2d1668"
format_stamp: "Formatted at 2026-08-07 22:35:36 on dist-test-slave-7kzw"
server_key: "a8a3ea4ff6ff4788bd81b009b9459e34"
server_key_iv: "820995a4b982e3daf4442d4f4250d160"
server_key_version: "encryptionkey@0"
I20260807 22:35:36.862376 344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:36.891204 344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:36.893211 344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:36.893396 344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:36.893652 344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:36.893990 344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:36.894048 344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:36.894088 344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:36.894119 344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:36.900348 344 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:35161
I20260807 22:35:36.900393 519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:35161 every 8 connection(s)
I20260807 22:35:36.900712 344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:36.904400 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 344
I20260807 22:35:36.904518 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:36.904719 29003 external_mini_cluster.cc:1428] Setting key 8289c065dcd56da297ab9a23936fb41e
Aug 07 22:35:36 dist-test-slave-7kzw krb5kdc[32457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142136, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:36.910923 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:36.901455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:37771 (local address 127.28.82.254:39847)
0807 22:35:36.901502 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:36.901506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:36.901702 (+ 196us) server_negotiation.cc:407] Connection header received
0807 22:35:36.902247 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:36.902248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:36.902259 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:36.902316 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:36.903332 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.903828 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.904936 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.905142 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.905230 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:36.907549 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:36.907551 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:36.907552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:36.907560 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:36.909396 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:36.909784 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:36.909785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:36.909785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:36.909861 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:36.910112 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:36.910113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:36.910114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:36.910301 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:36.910387 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:36.910733 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 22:35:36.910827 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:35:36.911105 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:36.901403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39847 (local address 127.28.82.194:37771)
0807 22:35:36.901568 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:36.901610 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:36.902152 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:36.902368 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:36.902371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:36.903052 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:36.903249 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:36.903254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.903914 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.903915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:36.904834 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:36.904838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:36.905239 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:36.905240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:36.905373 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:36.905885 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:36.905894 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:36.907441 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:36.909482 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:36.909488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:36.909490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:36.909703 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:36.909935 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:36.909935 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:36.909936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:36.910037 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:36.910386 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:36.910390 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:36.910562 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 22:35:36.910709 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 22:35:36.910866 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260807 22:35:36.911603 520 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39847
I20260807 22:35:36.911721 520 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:36.911942 520 heartbeater.cc:511] Master 127.28.82.254:39847 requested a full tablet report, sending...
I20260807 22:35:36.912441 32522 ts_manager.cc:194] Registered new tserver with Master: 69f0c3bbd7cb485b8496edea2e2d1668 (127.28.82.194:35161)
I20260807 22:35:36.912896 32522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:37771
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:36.933514 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39847
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:37.021692 526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:37.021890 526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:37.021914 526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:37.023058 526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:37.023099 526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:37.023125 526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:37.024468 526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:37.024662 526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:37.024837 526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:37.029973 537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:37.030045 526 server_base.cc:1034] running on GCE node
W20260807 22:35:37.029973 540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:37.030051 538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:37.030375 526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:37.030666 526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:37.031841 526 hybrid_clock.cc:648] HybridClock initialized: now 1786142137031826 us; error 25 us; skew 500 ppm
Aug 07 22:35:37 dist-test-slave-7kzw krb5kdc[32457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142137, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:37.034328 526 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:37.034807 526 webserver.cc:460] Webserver started at http://127.28.82.195:39493/ using document root <none> and password file <none>
I20260807 22:35:37.035051 526 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:37.035122 526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:37.035276 526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:37.036319 526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "be5c3ac1e67646668cc9ca1f0b6d2dd1"
format_stamp: "Formatted at 2026-08-07 22:35:37 on dist-test-slave-7kzw"
server_key: "1c28cf84a3a8d9d1f0bd8fe5a5a84916"
server_key_iv: "d7d76e4d9c20c468be0ca04fdeccbb0e"
server_key_version: "encryptionkey@0"
I20260807 22:35:37.036689 526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "be5c3ac1e67646668cc9ca1f0b6d2dd1"
format_stamp: "Formatted at 2026-08-07 22:35:37 on dist-test-slave-7kzw"
server_key: "1c28cf84a3a8d9d1f0bd8fe5a5a84916"
server_key_iv: "d7d76e4d9c20c468be0ca04fdeccbb0e"
server_key_version: "encryptionkey@0"
I20260807 22:35:37.038029 526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:37.038853 552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:37.039037 526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:37.039148 526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "be5c3ac1e67646668cc9ca1f0b6d2dd1"
format_stamp: "Formatted at 2026-08-07 22:35:37 on dist-test-slave-7kzw"
server_key: "1c28cf84a3a8d9d1f0bd8fe5a5a84916"
server_key_iv: "d7d76e4d9c20c468be0ca04fdeccbb0e"
server_key_version: "encryptionkey@0"
I20260807 22:35:37.039224 526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:37.048890 526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:37.051208 526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:37.051362 526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:37.051574 526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:37.051888 526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:37.051923 526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:37.051973 526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:37.052003 526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:37.057720 526 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:43817
I20260807 22:35:37.057798 719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:43817 every 8 connection(s)
I20260807 22:35:37.058116 526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:37.058955 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 526
I20260807 22:35:37.059056 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:37.059288 29003 external_mini_cluster.cc:1428] Setting key 3602e5ae8982f3fbda97a5cf8f82633c
Aug 07 22:35:37 dist-test-slave-7kzw krb5kdc[32457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142137, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:37.068090 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:37.058900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:40979 (local address 127.28.82.254:39847)
0807 22:35:37.058977 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:37.058980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:37.059194 (+ 214us) server_negotiation.cc:407] Connection header received
0807 22:35:37.059935 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:37.059936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:37.059948 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:37.060016 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:37.060969 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:37.061447 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:37.062384 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:37.062636 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:37.062692 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:37.064835 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:37.064837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:37.064837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:37.064845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:37.066614 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:37.067068 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:37.067070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:37.067070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:37.067136 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:37.067421 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:37.067422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:37.067422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:37.067551 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:37.067629 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:37.067901 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 22:35:37.067987 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:35:37.068244 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:37.058840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39847 (local address 127.28.82.195:40979)
0807 22:35:37.059056 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:37.059105 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:37.059827 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:37.060082 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:37.060085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:37.060707 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:37.060887 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:37.060891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:37.061526 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:37.061527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:37.062307 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:37.062309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:37.062731 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:37.062733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:37.062838 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:37.063255 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:37.063263 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:37.064722 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:37.066745 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:37.066751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:37.066753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:37.066973 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:37.067224 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:37.067224 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:37.067225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:37.067344 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:37.067631 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:37.067638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:37.067764 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 22:35:37.067866 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:35:37.068019 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":129,"threads_started":1}
I20260807 22:35:37.068744 720 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39847
I20260807 22:35:37.068830 720 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:37.069024 720 heartbeater.cc:511] Master 127.28.82.254:39847 requested a full tablet report, sending...
I20260807 22:35:37.069394 32522 ts_manager.cc:194] Registered new tserver with Master: be5c3ac1e67646668cc9ca1f0b6d2dd1 (127.28.82.195:43817)
I20260807 22:35:37.069747 32522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:40979
I20260807 22:35:37.071029 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:35:37.083268 32597 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:37.082245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.28.82.254:39847)
0807 22:35:37.082341 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:37.082343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:37.082453 (+ 110us) server_negotiation.cc:407] Connection header received
0807 22:35:37.082556 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:37.082558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:37.082569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:37.082642 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:37.083197 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38780: BlockingRecv error: recv got EOF from 127.0.0.1:38780 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 22:35:37.084233 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 32620
I20260807 22:35:37.102784 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 344
I20260807 22:35:37.115162 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 526
I20260807 22:35:37.124928 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 32474
2026-08-07T22:35:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3097 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:37 dist-test-slave-7kzw krb5kdc[735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:37 dist-test-slave-7kzw krb5kdc[735](info): set up 2 sockets
Aug 07 22:35:37 dist-test-slave-7kzw krb5kdc[735](info): commencing operation
krb5kdc: starting...
W20260807 22:35:38.355648 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:38 dist-test-slave-7kzw krb5kdc[735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:38.475591 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:44979
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:44979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:38.558578 751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:38.558866 751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:38.558904 751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:38.560410 751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:38.560453 751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:38.560467 751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:38.560479 751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:38.560536 751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:38.562026 751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:44979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:44979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:38.562285 751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:38.562582 751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:38.567462 756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:38.567457 759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:38.567477 757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:38.567770 751 server_base.cc:1034] running on GCE node
I20260807 22:35:38.568032 751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:38.568274 751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:38.569422 751 hybrid_clock.cc:648] HybridClock initialized: now 1786142138569399 us; error 37 us; skew 500 ppm
Aug 07 22:35:38 dist-test-slave-7kzw krb5kdc[735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142138, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:38.572273 751 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:38.572847 751 webserver.cc:460] Webserver started at http://127.28.82.254:39535/ using document root <none> and password file <none>
I20260807 22:35:38.573074 751 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:38.573123 751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:38.573278 751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:38.574329 751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "9b5179cb854345b3aebe58c3f63c1f36"
format_stamp: "Formatted at 2026-08-07 22:35:38 on dist-test-slave-7kzw"
server_key: "0d7236785005504259be05e04363614c"
server_key_iv: "203833106555fabe2a203e34387d352d"
server_key_version: "encryptionkey@0"
I20260807 22:35:38.574767 751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "9b5179cb854345b3aebe58c3f63c1f36"
format_stamp: "Formatted at 2026-08-07 22:35:38 on dist-test-slave-7kzw"
server_key: "0d7236785005504259be05e04363614c"
server_key_iv: "203833106555fabe2a203e34387d352d"
server_key_version: "encryptionkey@0"
I20260807 22:35:38.576208 751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:38.577020 771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:38.577201 751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:38.577312 751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "9b5179cb854345b3aebe58c3f63c1f36"
format_stamp: "Formatted at 2026-08-07 22:35:38 on dist-test-slave-7kzw"
server_key: "0d7236785005504259be05e04363614c"
server_key_iv: "203833106555fabe2a203e34387d352d"
server_key_version: "encryptionkey@0"
I20260807 22:35:38.577389 751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:38.595507 751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:38.597558 751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:38.597771 751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:38.602137 751 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:44979
I20260807 22:35:38.602181 848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:44979 every 8 connection(s)
I20260807 22:35:38.602736 751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:38.603394 849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:38.605854 849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36: Bootstrap starting.
I20260807 22:35:38.606503 849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:38.606819 849 log.cc:824] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:38.607517 849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36: No bootstrap required, opened a new log
I20260807 22:35:38.609143 849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5179cb854345b3aebe58c3f63c1f36" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44979 } }
I20260807 22:35:38.609284 849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:38.609308 849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5179cb854345b3aebe58c3f63c1f36, State: Initialized, Role: FOLLOWER
I20260807 22:35:38.609455 849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5179cb854345b3aebe58c3f63c1f36" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44979 } }
I20260807 22:35:38.609535 849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:38.609558 849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:38.609607 849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:38.610513 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 751
I20260807 22:35:38.610558 849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5179cb854345b3aebe58c3f63c1f36" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44979 } }
I20260807 22:35:38.610641 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:38.610675 849 leader_election.cc:302] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b5179cb854345b3aebe58c3f63c1f36; no voters:
I20260807 22:35:38.610890 29003 external_mini_cluster.cc:1428] Setting key 27581c527a2f7a6873942fca69494b66
I20260807 22:35:38.610882 849 leader_election.cc:288] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:38.610918 855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:38.611037 855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [term 1 LEADER]: Becoming Leader. State: Replica: 9b5179cb854345b3aebe58c3f63c1f36, State: Running, Role: LEADER
I20260807 22:35:38.611116 855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5179cb854345b3aebe58c3f63c1f36" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44979 } }
I20260807 22:35:38.611192 849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:38.611449 856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b5179cb854345b3aebe58c3f63c1f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5179cb854345b3aebe58c3f63c1f36" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44979 } } }
I20260807 22:35:38.611558 856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:38.611742 857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b5179cb854345b3aebe58c3f63c1f36. Latest consensus state: current_term: 1 leader_uuid: "9b5179cb854345b3aebe58c3f63c1f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b5179cb854345b3aebe58c3f63c1f36" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44979 } } }
I20260807 22:35:38.611822 857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:38.611965 863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:38.612545 863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:38.614177 863 catalog_manager.cc:1347] Generated new cluster ID: 9db63ea50348453b93c60ac9a26688dd
I20260807 22:35:38.614241 863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:35:38 dist-test-slave-7kzw krb5kdc[735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:38.623955 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:38.611567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.28.82.254:44979)
0807 22:35:38.612136 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:38.612146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:38.612181 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:35:38.612302 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:38.612309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:38.612379 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:38.613027 (+ 648us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:38.613845 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:38.615282 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:38.616347 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:38.617013 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:38.617131 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:38.620194 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:38.620198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:38.620198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:38.620212 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:38.622212 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:38.622701 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:38.622704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:38.622704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:38.622817 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:38.623046 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:38.623047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:38.623048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:38.623343 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:38.623534 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:38.623609 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 22:35:38.623740 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":70,"threads_started":1}
I20260807 22:35:38.624991 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:35:38.645630 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:38.641070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44660 (local address 127.28.82.254:44979)
0807 22:35:38.641118 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:38.641120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:38.641328 (+ 208us) server_negotiation.cc:407] Connection header received
0807 22:35:38.641385 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:38.641386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:38.641398 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:38.641454 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:38.641957 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:38.642804 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:38.643910 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:38.644238 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:38.644306 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:38.645081 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:38.645082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:38.645083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:38.645327 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:38.645348 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:38.645356 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:38.645427 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:38.645513 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 22:35:38.645539 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 22:35:38.646822 863 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:38.647464 863 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:38.660787 863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b5179cb854345b3aebe58c3f63c1f36: Generated new TSK 0
I20260807 22:35:38.661027 863 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:35:38.668869 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 751
2026-08-07T22:35:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1543 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:38 dist-test-slave-7kzw krb5kdc[891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:38 dist-test-slave-7kzw krb5kdc[891](info): set up 2 sockets
Aug 07 22:35:38 dist-test-slave-7kzw krb5kdc[891](info): commencing operation
krb5kdc: starting...
W20260807 22:35:39.896528 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:39 dist-test-slave-7kzw krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:40.023455 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:33721
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:33721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:40.107566 910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:40.107829 910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:40.107857 910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:40.108975 910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:40.109015 910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:40.109030 910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:40.109043 910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:40.109055 910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:40.110450 910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:33721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:33721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:40.110666 910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:40.110833 910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:40.115768 915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:40.115906 910 server_base.cc:1034] running on GCE node
W20260807 22:35:40.115792 919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:40.115805 916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:40.116289 910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:40.116595 910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:40.117776 910 hybrid_clock.cc:648] HybridClock initialized: now 1786142140117764 us; error 28 us; skew 500 ppm
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:40.120319 910 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:40.120786 910 webserver.cc:460] Webserver started at http://127.28.82.254:45759/ using document root <none> and password file <none>
I20260807 22:35:40.120970 910 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:40.121008 910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:40.121097 910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:40.122109 910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "525f0ca7f5ac459a8b4c2655a1c527e4"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "5e9fa1e445ffc218e5c2d49e268efb45"
server_key_iv: "e838a427f9bbfc6ca377f331941ca5f3"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.122426 910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "525f0ca7f5ac459a8b4c2655a1c527e4"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "5e9fa1e445ffc218e5c2d49e268efb45"
server_key_iv: "e838a427f9bbfc6ca377f331941ca5f3"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.123798 910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:40.124508 929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.124751 910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:40.124836 910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "525f0ca7f5ac459a8b4c2655a1c527e4"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "5e9fa1e445ffc218e5c2d49e268efb45"
server_key_iv: "e838a427f9bbfc6ca377f331941ca5f3"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.124892 910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:40.145021 910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:40.149570 910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:40.149760 910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:40.155210 910 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:33721
I20260807 22:35:40.155272 998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:33721 every 8 connection(s)
I20260807 22:35:40.155628 910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:40.156656 999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:40.159260 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4: Bootstrap starting.
I20260807 22:35:40.159901 999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:40.159906 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 910
I20260807 22:35:40.160001 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:40.160230 29003 external_mini_cluster.cc:1428] Setting key 74b58bce6fd5e832cfe8feb40ca4d16f
I20260807 22:35:40.160275 999 log.cc:824] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:40.161242 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4: No bootstrap required, opened a new log
I20260807 22:35:40.163772 999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "525f0ca7f5ac459a8b4c2655a1c527e4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33721 } }
I20260807 22:35:40.163944 999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:40.163988 999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 525f0ca7f5ac459a8b4c2655a1c527e4, State: Initialized, Role: FOLLOWER
I20260807 22:35:40.164160 999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "525f0ca7f5ac459a8b4c2655a1c527e4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33721 } }
I20260807 22:35:40.164263 999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:40.164309 999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:40.164359 999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:40.165001 999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "525f0ca7f5ac459a8b4c2655a1c527e4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33721 } }
I20260807 22:35:40.165117 999 leader_election.cc:302] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 525f0ca7f5ac459a8b4c2655a1c527e4; no voters:
I20260807 22:35:40.165283 999 leader_election.cc:288] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:40.165345 1004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:40.165522 1004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [term 1 LEADER]: Becoming Leader. State: Replica: 525f0ca7f5ac459a8b4c2655a1c527e4, State: Running, Role: LEADER
I20260807 22:35:40.165539 999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:40.165665 1004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "525f0ca7f5ac459a8b4c2655a1c527e4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33721 } }
I20260807 22:35:40.165992 1004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 525f0ca7f5ac459a8b4c2655a1c527e4. Latest consensus state: current_term: 1 leader_uuid: "525f0ca7f5ac459a8b4c2655a1c527e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "525f0ca7f5ac459a8b4c2655a1c527e4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33721 } } }
I20260807 22:35:40.166074 1004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:40.166258 1005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "525f0ca7f5ac459a8b4c2655a1c527e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "525f0ca7f5ac459a8b4c2655a1c527e4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33721 } } }
I20260807 22:35:40.166337 1005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:40.166499 1011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:40.167218 1011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:40.169026 1011 catalog_manager.cc:1347] Generated new cluster ID: 88c95e02175b465bbfecd5734f233ab0
I20260807 22:35:40.169081 1011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:40.172576 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:40.160762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.28.82.254:33721)
0807 22:35:40.160940 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:40.160949 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:40.160988 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:35:40.161608 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:40.161617 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:40.161680 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:40.161841 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:40.162442 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.163910 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.164800 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.165442 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.165895 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:40.168660 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:40.168664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:40.168665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:40.168676 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:40.170830 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.171254 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.171258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.171258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.171380 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.171600 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.171601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.171601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.171930 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:40.172111 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:40.172207 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 22:35:40.172351 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260807 22:35:40.179399 1011 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:40.179899 1011 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:40.188684 1011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 525f0ca7f5ac459a8b4c2655a1c527e4: Generated new TSK 0
I20260807 22:35:40.188902 1011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:40.219808 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33721
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:40.307633 1036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:40.307914 1036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:40.307940 1036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:40.309067 1036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:40.309108 1036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:40.309136 1036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:40.310652 1036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:40.310864 1036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:40.311062 1036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:35:40.316426 1036 server_base.cc:1034] running on GCE node
W20260807 22:35:40.316394 1046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:40.316385 1044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:40.316404 1043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:40.316793 1036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:40.317080 1036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:40.318252 1036 hybrid_clock.cc:648] HybridClock initialized: now 1786142140318245 us; error 40 us; skew 500 ppm
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:40.321010 1036 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:40.321453 1036 webserver.cc:460] Webserver started at http://127.28.82.193:41679/ using document root <none> and password file <none>
I20260807 22:35:40.321698 1036 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:40.321744 1036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:40.321911 1036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:40.323011 1036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "ce3e13f01d3546ce84acd0584b62e00f"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "7b0cfc396b870a61571ada3f56b96d4d"
server_key_iv: "d58e34421751f54f04c89b321abad073"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.323360 1036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "ce3e13f01d3546ce84acd0584b62e00f"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "7b0cfc396b870a61571ada3f56b96d4d"
server_key_iv: "d58e34421751f54f04c89b321abad073"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.324719 1036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:35:40.325443 1058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.325641 1036 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:40.325735 1036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "ce3e13f01d3546ce84acd0584b62e00f"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "7b0cfc396b870a61571ada3f56b96d4d"
server_key_iv: "d58e34421751f54f04c89b321abad073"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.325811 1036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:40.352422 1036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:40.354830 1036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:40.355005 1036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:40.355261 1036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:40.355582 1036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:40.355660 1036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.355684 1036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:40.355731 1036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.361476 1036 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:42353
I20260807 22:35:40.361569 1222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:42353 every 8 connection(s)
I20260807 22:35:40.361876 1036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:40.365724 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 1036
I20260807 22:35:40.365846 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:40.366125 29003 external_mini_cluster.cc:1428] Setting key 5126d61341ad204b7d30f0157c934767
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:40.371769 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:40.362663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:36247 (local address 127.28.82.254:33721)
0807 22:35:40.362730 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:40.362733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:40.362913 (+ 180us) server_negotiation.cc:407] Connection header received
0807 22:35:40.363415 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:40.363416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:40.363431 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:40.363509 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:40.364392 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.364915 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.366099 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.366448 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.366557 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:40.368781 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:40.368782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:40.368782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:40.368790 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:40.370245 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.370750 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.370751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.370751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.370815 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.371071 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.371072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.371072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.371223 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:40.371311 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:40.371584 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 22:35:40.371676 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:35:40.371968 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:40.362628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:33721 (local address 127.28.82.193:36247)
0807 22:35:40.362785 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:40.362828 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:40.363300 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:40.363558 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:40.363561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:40.364122 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:40.364301 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:40.364305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.365077 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.365078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:40.365978 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:40.365981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.366571 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.366573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:40.366717 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:40.367299 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:40.367307 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:40.368683 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:40.370350 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:40.370358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:40.370360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:40.370662 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:40.370884 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:40.370885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:40.370885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:40.371002 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:40.371302 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:40.371310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:40.371448 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 22:35:40.371556 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 22:35:40.371722 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260807 22:35:40.372467 1223 heartbeater.cc:347] Connected to a master server at 127.28.82.254:33721
I20260807 22:35:40.372597 1223 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:40.372807 1223 heartbeater.cc:511] Master 127.28.82.254:33721 requested a full tablet report, sending...
I20260807 22:35:40.373383 954 ts_manager.cc:194] Registered new tserver with Master: ce3e13f01d3546ce84acd0584b62e00f (127.28.82.193:42353)
I20260807 22:35:40.374087 954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:36247
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:40.393419 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33721
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:40.485456 1230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:40.485679 1230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:40.485704 1230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:40.487007 1230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:40.487062 1230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:40.487093 1230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:40.488716 1230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:40.488992 1230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:40.489272 1230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:40.494841 1238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:40.494843 1236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:40.494844 1235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:40.495222 1230 server_base.cc:1034] running on GCE node
I20260807 22:35:40.495409 1230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:40.495625 1230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:40.496784 1230 hybrid_clock.cc:648] HybridClock initialized: now 1786142140496768 us; error 30 us; skew 500 ppm
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:40.499581 1230 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:40.500077 1230 webserver.cc:460] Webserver started at http://127.28.82.194:45547/ using document root <none> and password file <none>
I20260807 22:35:40.500339 1230 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:40.500411 1230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:40.500560 1230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:40.501618 1230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "9ad96914fe8c4ed592d341d085b289e1"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "4a12aad815b0149cf35d7a173d9ab688"
server_key_iv: "43694b7796a508bfcd4cc891912669ba"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.502002 1230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "9ad96914fe8c4ed592d341d085b289e1"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "4a12aad815b0149cf35d7a173d9ab688"
server_key_iv: "43694b7796a508bfcd4cc891912669ba"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.503424 1230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:40.504179 1249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.504418 1230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:40.504501 1230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "9ad96914fe8c4ed592d341d085b289e1"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "4a12aad815b0149cf35d7a173d9ab688"
server_key_iv: "43694b7796a508bfcd4cc891912669ba"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.504594 1230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:40.526155 1230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:40.528923 1230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:40.529181 1230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:40.529610 1230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:40.529989 1230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:40.530025 1230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.530076 1230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:40.530102 1230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.537055 1230 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:38459
I20260807 22:35:40.537117 1435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:38459 every 8 connection(s)
I20260807 22:35:40.537421 1230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:40.539028 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 1230
I20260807 22:35:40.539163 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:40.539386 29003 external_mini_cluster.cc:1428] Setting key 603880f23f9a3eb6d977503d17b09ca2
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:40.549294 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:40.538151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:43613 (local address 127.28.82.254:33721)
0807 22:35:40.538204 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:40.538206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:40.538650 (+ 444us) server_negotiation.cc:407] Connection header received
0807 22:35:40.539499 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:40.539501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:40.539517 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:40.539611 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:40.540989 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.541477 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.542925 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.543138 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.543235 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:40.545766 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:40.545768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:40.545768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:40.545777 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:40.547793 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.548213 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.548214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.548215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.548299 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.548556 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.548557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.548558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.548728 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:40.548816 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:40.549087 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 22:35:40.549180 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:35:40.549311 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:40.538105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:33721 (local address 127.28.82.194:43613)
0807 22:35:40.538265 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:40.538316 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:40.539340 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:40.539699 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:40.539703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:40.540564 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:40.540857 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:40.540864 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.541584 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.541585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:40.542806 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:40.542809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.543246 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.543248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:40.543413 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:40.543944 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:40.543955 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:40.545661 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:40.547893 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:40.547900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:40.547902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:40.548128 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:40.548380 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:40.548380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:40.548381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:40.548484 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:40.548810 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:40.548815 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:40.548931 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 22:35:40.549024 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 22:35:40.549148 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260807 22:35:40.549998 1436 heartbeater.cc:347] Connected to a master server at 127.28.82.254:33721
I20260807 22:35:40.550127 1436 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:40.550359 1436 heartbeater.cc:511] Master 127.28.82.254:33721 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
I20260807 22:35:40.550896 954 ts_manager.cc:194] Registered new tserver with Master: 9ad96914fe8c4ed592d341d085b289e1 (127.28.82.194:38459)
I20260807 22:35:40.551291 954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:43613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:40.569561 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33721
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:40.654436 1441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:40.654703 1441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:40.654727 1441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:40.655824 1441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:40.655860 1441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:40.655886 1441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:40.657245 1441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:33721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:40.657439 1441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:40.657620 1441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:40.662708 1447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:40.662750 1449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:40.662715 1453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:40.663138 1441 server_base.cc:1034] running on GCE node
I20260807 22:35:40.663318 1441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:40.663511 1441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:40.664682 1441 hybrid_clock.cc:648] HybridClock initialized: now 1786142140664653 us; error 57 us; skew 500 ppm
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:40.667606 1441 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:40.668157 1441 webserver.cc:460] Webserver started at http://127.28.82.195:37849/ using document root <none> and password file <none>
I20260807 22:35:40.668411 1441 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:40.668488 1441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:40.668722 1441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:40.669883 1441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "664ca690e3bb486991d6733ec319157a"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "3de7d4cefa073ceb1d12bdb533c7320a"
server_key_iv: "b3bccd1ee48c8b8b709e8682437b84c5"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.670305 1441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "664ca690e3bb486991d6733ec319157a"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "3de7d4cefa073ceb1d12bdb533c7320a"
server_key_iv: "b3bccd1ee48c8b8b709e8682437b84c5"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.671842 1441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:40.672794 1462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.673044 1441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:40.673161 1441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "664ca690e3bb486991d6733ec319157a"
format_stamp: "Formatted at 2026-08-07 22:35:40 on dist-test-slave-7kzw"
server_key: "3de7d4cefa073ceb1d12bdb533c7320a"
server_key_iv: "b3bccd1ee48c8b8b709e8682437b84c5"
server_key_version: "encryptionkey@0"
I20260807 22:35:40.673250 1441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:40.683995 1441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:40.686151 1441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:40.686322 1441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:40.686586 1441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:40.686933 1441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:40.686991 1441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.687027 1441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:40.687073 1441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:40.693434 1441 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:45507
I20260807 22:35:40.693477 1627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:45507 every 8 connection(s)
I20260807 22:35:40.693840 1441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:40.694949 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 1441
I20260807 22:35:40.695030 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:40.695232 29003 external_mini_cluster.cc:1428] Setting key 17cdfee4d02d16c13738979f19ed1820
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:40.703828 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:40.694595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:44633 (local address 127.28.82.254:33721)
0807 22:35:40.694654 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:40.694655 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:40.694903 (+ 248us) server_negotiation.cc:407] Connection header received
0807 22:35:40.695605 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:40.695606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:40.695619 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:40.695662 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:40.696640 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.697156 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.698195 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.698411 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.698519 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:40.700728 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:40.700730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:40.700730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:40.700739 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:40.702162 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.702784 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.702786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.702786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.702858 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.703131 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.703132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.703132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.703263 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:40.703338 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:40.703635 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 22:35:40.703733 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:35:40.703999 1630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:40.694603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:33721 (local address 127.28.82.195:44633)
0807 22:35:40.694784 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:40.694822 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:40.695490 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:40.695744 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:40.695748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:40.696338 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:40.696530 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:40.696535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.697262 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.697263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:40.698080 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:40.698083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.698510 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.698511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:40.698657 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:40.699126 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:40.699135 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:40.700551 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:40.702278 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:40.702285 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:40.702288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:40.702675 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:40.702930 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:40.702931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:40.702931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:40.703054 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:40.703353 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:40.703359 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:40.703495 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 22:35:40.703595 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 22:35:40.703760 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260807 22:35:40.704452 1628 heartbeater.cc:347] Connected to a master server at 127.28.82.254:33721
I20260807 22:35:40.704562 1628 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:40.704775 1628 heartbeater.cc:511] Master 127.28.82.254:33721 requested a full tablet report, sending...
I20260807 22:35:40.705143 954 ts_manager.cc:194] Registered new tserver with Master: 664ca690e3bb486991d6733ec319157a (127.28.82.195:45507)
I20260807 22:35:40.705523 954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:44633
I20260807 22:35:40.706902 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142140, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:40.733178 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:40.725037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55412 (local address 127.28.82.254:33721)
0807 22:35:40.725159 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:40.725162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:40.725271 (+ 109us) server_negotiation.cc:407] Connection header received
0807 22:35:40.725391 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:40.725392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:40.725409 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:40.725491 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:40.726169 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.726755 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.727592 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:40.727791 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:40.727919 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:40.730693 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:40.730695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:40.730695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:40.730703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:40.732143 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.732505 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.732506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.732506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.732562 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:40.732736 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:40.732737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:40.732737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:40.732881 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:40.732961 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:40.733043 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 22:35:40.733105 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 22:35:40.733450 954 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55412
I20260807 22:35:40.733929 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 1036
I20260807 22:35:40.751672 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 1230
I20260807 22:35:40.760385 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 1441
I20260807 22:35:40.769841 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 910
2026-08-07T22:35:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2103 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[1644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[1644](info): set up 2 sockets
Aug 07 22:35:40 dist-test-slave-7kzw krb5kdc[1644](info): commencing operation
krb5kdc: starting...
W20260807 22:35:42.053625 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:42.179690 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43693
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:43693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:42.263685 1661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:42.263890 1661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:42.263933 1661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:42.265043 1661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:42.265077 1661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:42.265091 1661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:42.265105 1661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:42.265116 1661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:42.266647 1661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:43693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:42.266846 1661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:42.267022 1661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:42.271966 1670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:42.272006 1668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:42.271947 1672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:42.272331 1661 server_base.cc:1034] running on GCE node
I20260807 22:35:42.272653 1661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:42.272969 1661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:42.274185 1661 hybrid_clock.cc:648] HybridClock initialized: now 1786142142274150 us; error 57 us; skew 500 ppm
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:42.280488 1661 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:42.281016 1661 webserver.cc:460] Webserver started at http://127.28.82.254:45443/ using document root <none> and password file <none>
I20260807 22:35:42.281273 1661 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:42.281342 1661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:42.281504 1661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:42.282692 1661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "e876463a84734b5281f3f5cad6134e3d"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "3d570581afeed0e3d1507cd74f30af41"
server_key_iv: "a2a1b15cbf8ccd7b113c6887991ac355"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.283056 1661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "e876463a84734b5281f3f5cad6134e3d"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "3d570581afeed0e3d1507cd74f30af41"
server_key_iv: "a2a1b15cbf8ccd7b113c6887991ac355"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.284463 1661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:42.285318 1682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.285557 1661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:42.285719 1661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "e876463a84734b5281f3f5cad6134e3d"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "3d570581afeed0e3d1507cd74f30af41"
server_key_iv: "a2a1b15cbf8ccd7b113c6887991ac355"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.285804 1661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:42.297521 1661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:42.299670 1661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:42.299845 1661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:42.304602 1661 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:43693
I20260807 22:35:42.304667 1758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:43693 every 8 connection(s)
I20260807 22:35:42.305236 1661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:42.305423 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 1661
I20260807 22:35:42.305552 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:42.305766 29003 external_mini_cluster.cc:1428] Setting key 177d2fab85c4fac9fb7a56fd651a856b
I20260807 22:35:42.306128 1759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:42.309111 1759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d: Bootstrap starting.
I20260807 22:35:42.309783 1759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:42.310102 1759 log.cc:824] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:42.310830 1759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d: No bootstrap required, opened a new log
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:42.313323 1759 raft_consensus.cc:348] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e876463a84734b5281f3f5cad6134e3d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43693 } }
I20260807 22:35:42.313486 1759 raft_consensus.cc:374] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:42.313552 1759 raft_consensus.cc:729] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e876463a84734b5281f3f5cad6134e3d, State: Initialized, Role: FOLLOWER
I20260807 22:35:42.313728 1759 consensus_queue.cc:260] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e876463a84734b5281f3f5cad6134e3d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43693 } }
I20260807 22:35:42.313835 1759 raft_consensus.cc:388] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:42.313869 1759 raft_consensus.cc:482] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:42.313922 1759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:42.314679 1759 raft_consensus.cc:504] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e876463a84734b5281f3f5cad6134e3d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43693 } }
I20260807 22:35:42.314790 1759 leader_election.cc:302] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e876463a84734b5281f3f5cad6134e3d; no voters:
I20260807 22:35:42.314949 1759 leader_election.cc:288] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:42.315011 1764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:42.315186 1764 raft_consensus.cc:686] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [term 1 LEADER]: Becoming Leader. State: Replica: e876463a84734b5281f3f5cad6134e3d, State: Running, Role: LEADER
I20260807 22:35:42.315348 1764 consensus_queue.cc:237] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e876463a84734b5281f3f5cad6134e3d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43693 } }
I20260807 22:35:42.315222 1759 sys_catalog.cc:564] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:42.315719 1766 sys_catalog.cc:455] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e876463a84734b5281f3f5cad6134e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e876463a84734b5281f3f5cad6134e3d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43693 } } }
I20260807 22:35:42.315805 1766 sys_catalog.cc:458] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:42.315737 1765 sys_catalog.cc:455] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e876463a84734b5281f3f5cad6134e3d. Latest consensus state: current_term: 1 leader_uuid: "e876463a84734b5281f3f5cad6134e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e876463a84734b5281f3f5cad6134e3d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43693 } } }
I20260807 22:35:42.315848 1765 sys_catalog.cc:458] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:42.315995 1768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:42.316515 1768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:42.317103 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.306333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.28.82.254:43693)
0807 22:35:42.306554 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:42.306564 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:42.306598 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:35:42.307325 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:42.307333 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:42.307382 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:42.307524 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:42.308190 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.309161 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.309894 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.310343 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.310467 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.313098 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:42.313103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:42.313103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:42.313120 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:42.315240 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.315671 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.315673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.315674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.315795 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.316156 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.316156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.316157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.316468 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:42.316630 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:42.316761 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 22:35:42.316902 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":88,"threads_started":1}
I20260807 22:35:42.318257 1768 catalog_manager.cc:1347] Generated new cluster ID: bb067b25f3a24971ab030307713c6d59
I20260807 22:35:42.318310 1768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:42.338096 1768 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:42.338721 1768 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:42.343052 1768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e876463a84734b5281f3f5cad6134e3d: Generated new TSK 0
I20260807 22:35:42.343272 1768 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:42.373028 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43693
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:42.456104 1788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:42.456307 1788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:42.456329 1788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:42.457393 1788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:42.457430 1788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:42.457455 1788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:42.459046 1788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:42.459277 1788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:42.459444 1788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:42.464497 1798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:42.464488 1797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:42.464486 1800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:42.464843 1788 server_base.cc:1034] running on GCE node
I20260807 22:35:42.465060 1788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:42.465312 1788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:42.466527 1788 hybrid_clock.cc:648] HybridClock initialized: now 1786142142466512 us; error 26 us; skew 500 ppm
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:42.469211 1788 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:42.469666 1788 webserver.cc:460] Webserver started at http://127.28.82.193:37923/ using document root <none> and password file <none>
I20260807 22:35:42.469844 1788 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:42.469884 1788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:42.469970 1788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:42.471007 1788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "9f2984be11854678ba440cdd47306cc1"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "4f7865c41ae028c0595eb2b5ca4e7c40"
server_key_iv: "5c35c3bbcca4716421b7804f8b3eff1d"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.471321 1788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "9f2984be11854678ba440cdd47306cc1"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "4f7865c41ae028c0595eb2b5ca4e7c40"
server_key_iv: "5c35c3bbcca4716421b7804f8b3eff1d"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.472604 1788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:42.473364 1810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.473601 1788 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:42.473691 1788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "9f2984be11854678ba440cdd47306cc1"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "4f7865c41ae028c0595eb2b5ca4e7c40"
server_key_iv: "5c35c3bbcca4716421b7804f8b3eff1d"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.473747 1788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:42.481165 1788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:42.483299 1788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:42.483418 1788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:42.483636 1788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:42.483916 1788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:42.483954 1788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.483976 1788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:42.483989 1788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.490141 1788 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:44743
I20260807 22:35:42.490211 1974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:44743 every 8 connection(s)
I20260807 22:35:42.490612 1788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:42.498755 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 1788
I20260807 22:35:42.498868 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:42.499106 29003 external_mini_cluster.cc:1428] Setting key 65524fee30ca02ea7374989fe064566a
I20260807 22:35:42.501107 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.491374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:52583 (local address 127.28.82.254:43693)
0807 22:35:42.491448 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:42.491450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:42.491676 (+ 226us) server_negotiation.cc:407] Connection header received
0807 22:35:42.492667 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:42.492669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:42.492682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:42.492744 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:42.493413 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.493941 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.495137 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.495353 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.495526 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.497799 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:42.497800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:42.497800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:42.497808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:42.499481 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.500012 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.500013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.500014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.500087 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.500374 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.500375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.500375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.500504 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:42.500585 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:42.500908 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 22:35:42.501009 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:35:42.501320 1978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.491369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43693 (local address 127.28.82.193:52583)
0807 22:35:42.491548 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:42.491584 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:42.492507 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:42.492831 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:42.492834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:42.493106 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:42.493314 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:42.493318 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.494059 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.494060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:42.495025 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:42.495028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.495536 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.495537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:42.495649 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.496210 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:42.496220 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:42.497702 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:42.499575 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:42.499585 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:42.499587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:42.499907 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:42.500167 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:42.500168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:42.500169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:42.500294 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:42.500575 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:42.500583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:42.500739 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 22:35:42.500873 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 22:35:42.501054 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260807 22:35:42.501799 1975 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43693
I20260807 22:35:42.501930 1975 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:42.502131 1975 heartbeater.cc:511] Master 127.28.82.254:43693 requested a full tablet report, sending...
I20260807 22:35:42.502727 1706 ts_manager.cc:194] Registered new tserver with Master: 9f2984be11854678ba440cdd47306cc1 (127.28.82.193:44743)
I20260807 22:35:42.503376 1706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:52583
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:42.529322 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43693
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:42.614295 1981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:42.614674 1981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:42.614753 1981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:42.616026 1981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:42.616086 1981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:42.616128 1981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:42.617786 1981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:42.618075 1981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:42.618335 1981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:42.623512 1989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:42.623517 1990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:42.623553 1993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:42.623970 1981 server_base.cc:1034] running on GCE node
I20260807 22:35:42.624239 1981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:42.624516 1981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:42.625717 1981 hybrid_clock.cc:648] HybridClock initialized: now 1786142142625706 us; error 37 us; skew 500 ppm
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:42.628511 1981 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:42.629089 1981 webserver.cc:460] Webserver started at http://127.28.82.194:39351/ using document root <none> and password file <none>
I20260807 22:35:42.629339 1981 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:42.629415 1981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:42.629594 1981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:42.630811 1981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "913012b8c3e048388652f12abddf7969"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "bad87a0af2f0dd51895e9d58049e10cc"
server_key_iv: "9581740ae4651800fa6287c7aa76cddf"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.631233 1981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "913012b8c3e048388652f12abddf7969"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "bad87a0af2f0dd51895e9d58049e10cc"
server_key_iv: "9581740ae4651800fa6287c7aa76cddf"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.632763 1981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:42.633596 2003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.633821 1981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:42.633936 1981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "913012b8c3e048388652f12abddf7969"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "bad87a0af2f0dd51895e9d58049e10cc"
server_key_iv: "9581740ae4651800fa6287c7aa76cddf"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.634016 1981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:42.642234 1981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:42.644629 1981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:42.644829 1981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:42.645123 1981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:42.645480 1981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:42.645540 1981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.645576 1981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:42.645601 1981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.651791 1981 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:33893
I20260807 22:35:42.651836 2157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:33893 every 8 connection(s)
I20260807 22:35:42.652179 1981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:42.655619 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 1981
I20260807 22:35:42.655740 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:42.655946 29003 external_mini_cluster.cc:1428] Setting key 90f25020d8daf77ba374b7722eb43ae6
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:42.663432 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.652922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43693 (local address 127.28.82.194:55075)
0807 22:35:42.653097 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:42.653141 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:42.654124 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:42.654404 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:42.654408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:42.654726 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:42.654926 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:42.654930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.655891 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.655892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:42.656945 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:42.656948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.657468 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.657469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:42.657599 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.658210 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:42.658219 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:42.659980 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:42.661982 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:42.661988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:42.661990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:42.662225 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:42.662445 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:42.662446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:42.662446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:42.662603 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:42.662915 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:42.662920 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:42.663041 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 22:35:42.663133 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:35:42.663257 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260807 22:35:42.663443 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.652947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:55075 (local address 127.28.82.254:43693)
0807 22:35:42.653031 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:42.653034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:42.653231 (+ 197us) server_negotiation.cc:407] Connection header received
0807 22:35:42.654256 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:42.654258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:42.654272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:42.654332 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:42.655032 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.655794 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.657035 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.657347 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.657428 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.660074 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:42.660075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:42.660076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:42.660085 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:42.661883 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.662300 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.662301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.662302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.662372 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.662675 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.662676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.662677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.662838 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:42.662924 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:42.663200 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 22:35:42.663300 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:35:42.663954 2158 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43693
I20260807 22:35:42.664083 2158 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:42.664321 2158 heartbeater.cc:511] Master 127.28.82.254:43693 requested a full tablet report, sending...
I20260807 22:35:42.664762 1706 ts_manager.cc:194] Registered new tserver with Master: 913012b8c3e048388652f12abddf7969 (127.28.82.194:33893)
I20260807 22:35:42.665320 1706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:55075
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:42.684886 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43693
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:42.771571 2166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:42.771826 2166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:42.771854 2166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:42.773191 2166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:42.773244 2166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:42.773274 2166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:42.774999 2166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:42.775275 2166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:42.775535 2166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:42.780930 2171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:42.780982 2172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:42.781162 2177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:42.781251 2166 server_base.cc:1034] running on GCE node
I20260807 22:35:42.781507 2166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:42.781759 2166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:42.782909 2166 hybrid_clock.cc:648] HybridClock initialized: now 1786142142782907 us; error 44 us; skew 500 ppm
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:42.785445 2166 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:42.785976 2166 webserver.cc:460] Webserver started at http://127.28.82.195:40821/ using document root <none> and password file <none>
I20260807 22:35:42.786211 2166 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:42.786281 2166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:42.786437 2166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:42.787572 2166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "ed4b6972ddfe4ad7ac940d664598a5ca"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "a123bfb2fa636bf7d483e0995a43251c"
server_key_iv: "0a605b30cc1fe643bc39b286dc4379f5"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.787963 2166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "ed4b6972ddfe4ad7ac940d664598a5ca"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "a123bfb2fa636bf7d483e0995a43251c"
server_key_iv: "0a605b30cc1fe643bc39b286dc4379f5"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.789429 2166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:42.790342 2186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.790659 2166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:42.790776 2166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "ed4b6972ddfe4ad7ac940d664598a5ca"
format_stamp: "Formatted at 2026-08-07 22:35:42 on dist-test-slave-7kzw"
server_key: "a123bfb2fa636bf7d483e0995a43251c"
server_key_iv: "0a605b30cc1fe643bc39b286dc4379f5"
server_key_version: "encryptionkey@0"
I20260807 22:35:42.790868 2166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:42.813171 2166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:42.815594 2166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:42.815773 2166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:42.816094 2166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:42.816413 2166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:42.816499 2166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.816555 2166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:42.816583 2166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:42.823017 2166 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:38113
I20260807 22:35:42.823083 2334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:38113 every 8 connection(s)
I20260807 22:35:42.823436 2166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:42.831580 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 2166
I20260807 22:35:42.831701 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:42.831928 29003 external_mini_cluster.cc:1428] Setting key 8b099598d04941ddfea9cab370690f36
I20260807 22:35:42.832772 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.824201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:38043 (local address 127.28.82.254:43693)
0807 22:35:42.824279 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:42.824281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:42.824456 (+ 175us) server_negotiation.cc:407] Connection header received
0807 22:35:42.825396 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:42.825397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:42.825408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:42.825470 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:42.826058 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.826618 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.827577 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.827755 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.827804 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.829830 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:42.829831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:42.829831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:42.829838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:42.831183 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.831588 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.831589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.831589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.831654 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.831966 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.831982 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.831982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.832149 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:42.832221 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:42.832553 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 22:35:42.832637 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:35:42.832952 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.824170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43693 (local address 127.28.82.195:38043)
0807 22:35:42.824331 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:42.824374 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:42.825307 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:42.825537 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:42.825540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:42.825802 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:42.825988 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:42.825992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.826698 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.826699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:42.827505 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:42.827507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.827823 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.827824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:42.827919 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.828369 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:42.828377 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:42.829747 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:42.831263 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:42.831268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:42.831270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:42.831511 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:42.831746 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:42.831747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:42.831747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:42.831891 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:42.832229 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:42.832236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:42.832400 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 22:35:42.832518 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 22:35:42.832703 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260807 22:35:42.833463 2337 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43693
I20260807 22:35:42.833576 2337 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:42.833799 2337 heartbeater.cc:511] Master 127.28.82.254:43693 requested a full tablet report, sending...
I20260807 22:35:42.834216 1706 ts_manager.cc:194] Registered new tserver with Master: ed4b6972ddfe4ad7ac940d664598a5ca (127.28.82.195:38113)
I20260807 22:35:42.834784 1706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:38043
I20260807 22:35:42.843886 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:42.868198 2354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.860360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.28.82.193:44743)
0807 22:35:42.860520 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:42.860528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:42.860544 (+ 16us) server_negotiation.cc:407] Connection header received
0807 22:35:42.860572 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:42.860574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:42.860589 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:42.860664 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:42.861293 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.862031 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.862789 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.863102 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.863164 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.865299 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:42.865307 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:42.865308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:42.865315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:42.867082 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.867434 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.867437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.867438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.867534 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.867739 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.867740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.867740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.867882 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:42.868010 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:42.868046 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 22:35:42.868116 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[1644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:42.876349 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:42.869675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60154 (local address 127.28.82.254:43693)
0807 22:35:42.869740 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:42.869742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:42.869750 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:42.869773 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:42.869774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:42.869785 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:42.869869 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:42.870331 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.870883 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.871559 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:42.871765 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:42.871840 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:42.873869 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:42.873871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:42.873872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:42.873879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:42.875108 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.875473 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.875475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.875475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.875536 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:42.875738 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:42.875739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:42.875739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:42.875952 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:42.876066 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:42.876163 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 22:35:42.876245 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 22:35:42.876608 1706 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60154
I20260807 22:35:42.877035 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 1788
I20260807 22:35:42.890578 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 1981
I20260807 22:35:42.900780 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 2166
I20260807 22:35:42.906386 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 1661
2026-08-07T22:35:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2132 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[2359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[2359](info): set up 2 sockets
Aug 07 22:35:42 dist-test-slave-7kzw krb5kdc[2359](info): commencing operation
krb5kdc: starting...
W20260807 22:35:44.123133 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:44.245854 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39567
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:44.327504 2377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:44.327714 2377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:44.327737 2377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:44.328821 2377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:44.328855 2377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:44.328868 2377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:44.328881 2377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:44.328935 2377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:44.330415 2377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:44.330675 2377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:44.330860 2377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:44.335683 2383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:44.335700 2385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:44.335728 2388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:44.335739 2377 server_base.cc:1034] running on GCE node
I20260807 22:35:44.336198 2377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:44.336449 2377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:44.337636 2377 hybrid_clock.cc:648] HybridClock initialized: now 1786142144337621 us; error 32 us; skew 500 ppm
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:44.340327 2377 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:44.340809 2377 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:44.340910 2377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:44.341081 2377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:44.342293 2377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "18808635d1914cc78722000cd037d68f"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "0326e8ceb63c2f654fc1bebc34426742"
server_key_iv: "cc8b58dd3f0ebe63ea4ad381d672e644"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.342737 2377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "18808635d1914cc78722000cd037d68f"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "0326e8ceb63c2f654fc1bebc34426742"
server_key_iv: "cc8b58dd3f0ebe63ea4ad381d672e644"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.344166 2377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 22:35:44.345069 2397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.345266 2377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:44.345388 2377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "18808635d1914cc78722000cd037d68f"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "0326e8ceb63c2f654fc1bebc34426742"
server_key_iv: "cc8b58dd3f0ebe63ea4ad381d672e644"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.345467 2377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:44.365854 2377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:44.367990 2377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:44.368165 2377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:44.372548 2377 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:39567
I20260807 22:35:44.372581 2466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:39567 every 8 connection(s)
I20260807 22:35:44.372870 2377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:44.373827 2468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:44.376221 2468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f: Bootstrap starting.
I20260807 22:35:44.376839 2468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:44.377171 2468 log.cc:824] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:44.377851 2468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f: No bootstrap required, opened a new log
I20260807 22:35:44.379560 2468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18808635d1914cc78722000cd037d68f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39567 } }
I20260807 22:35:44.379698 2468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:44.379722 2468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18808635d1914cc78722000cd037d68f, State: Initialized, Role: FOLLOWER
I20260807 22:35:44.379895 2468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18808635d1914cc78722000cd037d68f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39567 } }
I20260807 22:35:44.379958 2468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:44.379980 2468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:44.380033 2468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:44.380811 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 2377
I20260807 22:35:44.380928 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:44.380977 2468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18808635d1914cc78722000cd037d68f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39567 } }
I20260807 22:35:44.381091 2468 leader_election.cc:302] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18808635d1914cc78722000cd037d68f; no voters:
I20260807 22:35:44.381141 29003 external_mini_cluster.cc:1428] Setting key 290cc2e49c16054f65eb94961e684d68
I20260807 22:35:44.381279 2468 leader_election.cc:288] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:44.381352 2471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:44.381547 2471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [term 1 LEADER]: Becoming Leader. State: Replica: 18808635d1914cc78722000cd037d68f, State: Running, Role: LEADER
I20260807 22:35:44.381590 2468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:44.381681 2471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18808635d1914cc78722000cd037d68f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39567 } }
I20260807 22:35:44.382037 2472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18808635d1914cc78722000cd037d68f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18808635d1914cc78722000cd037d68f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39567 } } }
I20260807 22:35:44.382159 2472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:44.382387 2472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18808635d1914cc78722000cd037d68f. Latest consensus state: current_term: 1 leader_uuid: "18808635d1914cc78722000cd037d68f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18808635d1914cc78722000cd037d68f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39567 } } }
I20260807 22:35:44.382484 2484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:44.382545 2472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:44.383211 2484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:44.384873 2484 catalog_manager.cc:1347] Generated new cluster ID: 1263de937b2846cea25e118e21ad3733
I20260807 22:35:44.384928 2484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:44.392014 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.381823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.28.82.254:39567)
0807 22:35:44.381988 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:44.381999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:44.382182 (+ 183us) server_negotiation.cc:407] Connection header received
0807 22:35:44.382324 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:44.382333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:44.382392 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:44.382586 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:44.383687 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.384981 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.385732 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.386167 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.386287 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.388597 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:44.388600 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:44.388601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:44.388612 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:44.390340 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.390800 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.390803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.390804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.390904 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.391120 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.391121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.391122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.391365 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:44.391537 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:44.391609 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 22:35:44.391853 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260807 22:35:44.394595 2484 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:44.395123 2484 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:44.403985 2484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f: Generated new TSK 0
I20260807 22:35:44.404178 2484 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:44.439127 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39567
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:44.525789 2502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:44.526005 2502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:44.526029 2502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:44.527160 2502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:44.527196 2502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:44.527223 2502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:44.528699 2502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:44.528882 2502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:44.529064 2502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:44.534102 2511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:44.534037 2514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:44.534379 2502 server_base.cc:1034] running on GCE node
W20260807 22:35:44.534037 2510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:44.534675 2502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:44.534952 2502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:44.536093 2502 hybrid_clock.cc:648] HybridClock initialized: now 1786142144536083 us; error 28 us; skew 500 ppm
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:44.538669 2502 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:44.539000 2502 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:44.539048 2502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:44.539201 2502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:44.540220 2502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "f287c578a409494eaf1e70ce1a5d42f0"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "6b7c43baabc6fdca2d354e91b622626a"
server_key_iv: "802dd21c7dcf3d47783269d53bc0f125"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.540575 2502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "f287c578a409494eaf1e70ce1a5d42f0"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "6b7c43baabc6fdca2d354e91b622626a"
server_key_iv: "802dd21c7dcf3d47783269d53bc0f125"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.541958 2502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:44.542733 2526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.542906 2502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:44.543009 2502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "f287c578a409494eaf1e70ce1a5d42f0"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "6b7c43baabc6fdca2d354e91b622626a"
server_key_iv: "802dd21c7dcf3d47783269d53bc0f125"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.543080 2502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:44.556501 2502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:44.559060 2502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:44.559233 2502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:44.559480 2502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:44.559829 2502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:44.559887 2502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.559935 2502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:44.559983 2502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.566440 2502 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:40685
I20260807 22:35:44.566519 2675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:40685 every 8 connection(s)
I20260807 22:35:44.566845 2502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:44.575524 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 2502
I20260807 22:35:44.575654 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:44.575873 29003 external_mini_cluster.cc:1428] Setting key 4156699081ecd7e0071f64bb9c084840
I20260807 22:35:44.576802 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.567700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:39331 (local address 127.28.82.254:39567)
0807 22:35:44.567820 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:44.567822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:44.567982 (+ 160us) server_negotiation.cc:407] Connection header received
0807 22:35:44.568515 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:44.568516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:44.568533 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:44.568586 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:44.569547 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.570049 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.571108 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.571327 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.571425 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.573609 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:44.573611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:44.573611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:44.573619 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:44.575160 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.575597 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.575598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.575599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.575673 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.575986 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.575987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.575988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.576119 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:44.576228 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:44.576581 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 22:35:44.576707 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 22:35:44.576965 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.567612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39567 (local address 127.28.82.193:39331)
0807 22:35:44.567848 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:44.567896 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:44.568401 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:44.568704 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:44.568707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:44.568955 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:44.569439 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.569443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.570149 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.570151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.571011 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.571014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.571452 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.571453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.571569 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.572061 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:44.572069 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:44.573496 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:44.575253 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:44.575260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:44.575262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:44.575514 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:44.575758 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:44.575759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:44.575759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:44.575886 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:44.576215 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:44.576221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:44.576395 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 22:35:44.576512 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 22:35:44.576737 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":108,"threads_started":1}
I20260807 22:35:44.577474 2676 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39567
I20260807 22:35:44.577589 2676 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:44.577754 2676 heartbeater.cc:511] Master 127.28.82.254:39567 requested a full tablet report, sending...
I20260807 22:35:44.578289 2420 ts_manager.cc:194] Registered new tserver with Master: f287c578a409494eaf1e70ce1a5d42f0 (127.28.82.193:40685)
I20260807 22:35:44.579242 2420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:39331
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:44.605399 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39567
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:44.689843 2682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:44.690057 2682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:44.690081 2682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:44.691180 2682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:44.691216 2682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:44.691242 2682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:44.692715 2682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:44.692904 2682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:44.693086 2682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:44.698021 2689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:44.698100 2691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:44.698033 2694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:44.698172 2682 server_base.cc:1034] running on GCE node
I20260807 22:35:44.698457 2682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:44.698760 2682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:44.699925 2682 hybrid_clock.cc:648] HybridClock initialized: now 1786142144699893 us; error 54 us; skew 500 ppm
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:44.702724 2682 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:44.703119 2682 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:44.703195 2682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:44.703336 2682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:44.704388 2682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "26be0da92c2648c99941411b9a961982"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "f98cca98cbcbe824ce86c3f511228236"
server_key_iv: "d39ffe42eae543c7a368ced193135a95"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.704770 2682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "26be0da92c2648c99941411b9a961982"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "f98cca98cbcbe824ce86c3f511228236"
server_key_iv: "d39ffe42eae543c7a368ced193135a95"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.706094 2682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:44.706991 2709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.707167 2682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.707283 2682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "26be0da92c2648c99941411b9a961982"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "f98cca98cbcbe824ce86c3f511228236"
server_key_iv: "d39ffe42eae543c7a368ced193135a95"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.707361 2682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:44.718303 2682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:44.720275 2682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:44.720456 2682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:44.720700 2682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:44.721027 2682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:44.721086 2682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.721120 2682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:44.721153 2682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.727149 2682 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:44619
I20260807 22:35:44.727187 2867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:44619 every 8 connection(s)
I20260807 22:35:44.727476 2682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:44.731060 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 2682
I20260807 22:35:44.731154 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:44.731348 29003 external_mini_cluster.cc:1428] Setting key d3a6e0b2e1e1c20ee4ace9df3b08a81c
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:44.737404 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.728225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:60227 (local address 127.28.82.254:39567)
0807 22:35:44.728271 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:44.728273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:44.728482 (+ 209us) server_negotiation.cc:407] Connection header received
0807 22:35:44.728991 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:44.728993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:44.729005 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:44.729059 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:44.729941 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.730560 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.731680 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.731944 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.732021 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.734250 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:44.734252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:44.734253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:44.734262 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:44.735835 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.736345 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.736346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.736346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.736405 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.736740 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.736740 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.736741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.736854 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:44.736940 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:44.737227 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 22:35:44.737313 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:35:44.737588 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.728202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39567 (local address 127.28.82.194:60227)
0807 22:35:44.728357 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:44.728391 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:44.728892 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:44.729125 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:44.729129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:44.729376 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:44.729845 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.729850 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.730663 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.730664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.731586 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.731588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.732037 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.732038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.732181 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.732752 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:44.732761 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:44.734149 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:44.735919 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:44.735927 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:44.735929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:44.736264 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:44.736476 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:44.736477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:44.736478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:44.736662 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:44.736944 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:44.736951 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:44.737078 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 22:35:44.737201 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:35:44.737356 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260807 22:35:44.738093 2868 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39567
I20260807 22:35:44.738183 2868 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:44.738336 2868 heartbeater.cc:511] Master 127.28.82.254:39567 requested a full tablet report, sending...
I20260807 22:35:44.738785 2420 ts_manager.cc:194] Registered new tserver with Master: 26be0da92c2648c99941411b9a961982 (127.28.82.194:44619)
I20260807 22:35:44.739316 2420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:60227
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:44.761607 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39567
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:44.848767 2878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:44.849021 2878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:44.849054 2878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:44.850168 2878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:44.850206 2878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:44.850232 2878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:44.851761 2878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:44.851962 2878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:44.852137 2878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:44.857252 2888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:44.857290 2884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:44.857272 2885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:44.857570 2878 server_base.cc:1034] running on GCE node
I20260807 22:35:44.857795 2878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:44.858023 2878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:44.859208 2878 hybrid_clock.cc:648] HybridClock initialized: now 1786142144859188 us; error 36 us; skew 500 ppm
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:44.861826 2878 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:44.862222 2878 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:44.862300 2878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:44.862457 2878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:44.863601 2878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "abef618bd2b049ebb1b2107dcc5a9148"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "dfe31629e65e908196602e0824fbef2e"
server_key_iv: "300f31c9d665a35cdff43e0a33417821"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.863994 2878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "abef618bd2b049ebb1b2107dcc5a9148"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "dfe31629e65e908196602e0824fbef2e"
server_key_iv: "300f31c9d665a35cdff43e0a33417821"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.866098 2878 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 22:35:44.866971 2897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.867173 2878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.867302 2878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "abef618bd2b049ebb1b2107dcc5a9148"
format_stamp: "Formatted at 2026-08-07 22:35:44 on dist-test-slave-7kzw"
server_key: "dfe31629e65e908196602e0824fbef2e"
server_key_iv: "300f31c9d665a35cdff43e0a33417821"
server_key_version: "encryptionkey@0"
I20260807 22:35:44.867379 2878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:44.884735 2878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:44.887110 2878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:44.887287 2878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:44.887511 2878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:44.887838 2878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:44.887880 2878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.887928 2878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:44.887953 2878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:44.893566 2878 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:42913
I20260807 22:35:44.893682 3059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:42913 every 8 connection(s)
I20260807 22:35:44.893922 2878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:44.897284 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 2878
I20260807 22:35:44.897364 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:44.897544 29003 external_mini_cluster.cc:1428] Setting key f5c93c03cc74baabbc4a04220ed1c504
Aug 07 22:35:44 dist-test-slave-7kzw krb5kdc[2359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142144, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:44.903679 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.894754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:52909 (local address 127.28.82.254:39567)
0807 22:35:44.894818 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:44.894820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:44.895005 (+ 185us) server_negotiation.cc:407] Connection header received
0807 22:35:44.895535 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:44.895537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:44.895550 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:44.895601 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:44.896521 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.897024 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.898379 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.898680 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.898764 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.900948 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:44.900949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:44.900950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:44.900958 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:44.902215 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.902694 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.902695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.902696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.902764 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.903020 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.903021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.903021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.903150 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:44.903223 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:44.903497 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 22:35:44.903580 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:35:44.903859 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.894715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39567 (local address 127.28.82.195:52909)
0807 22:35:44.894879 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:44.894925 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:44.895420 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:44.895674 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:44.895678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:44.895938 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:44.896418 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.896422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.897122 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.897123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.898268 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.898271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.898768 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.898769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.898920 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.899446 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:44.899455 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:44.900848 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:44.902300 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:44.902307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:44.902310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:44.902597 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:44.902831 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:44.902832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:44.902833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:44.902948 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:44.903226 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:44.903233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:44.903364 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 22:35:44.903471 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 22:35:44.903624 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260807 22:35:44.904332 3060 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39567
I20260807 22:35:44.904465 3060 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:44.904618 3060 heartbeater.cc:511] Master 127.28.82.254:39567 requested a full tablet report, sending...
I20260807 22:35:44.905018 2420 ts_manager.cc:194] Registered new tserver with Master: abef618bd2b049ebb1b2107dcc5a9148 (127.28.82.195:42913)
I20260807 22:35:44.905398 2420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:52909
I20260807 22:35:44.909332 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:35:44.916726 2475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.910638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.28.82.254:39567)
0807 22:35:44.910684 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:44.910686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:44.910790 (+ 104us) server_negotiation.cc:407] Connection header received
0807 22:35:44.910837 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:44.910838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:44.910845 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:44.910893 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:44.911499 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.911950 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.912747 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.912929 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.912991 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.914206 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:44.914207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:44.914207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:44.914213 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:44.915540 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.916052 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.916053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.916053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.916111 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:44.916285 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:44.916286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:44.916286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:44.916417 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:44.916492 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:44.916608 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 22:35:44.916668 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:35:44.918397 2420 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:44.920043 2420 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:44.926213 3080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.922615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38200 (local address 127.28.82.195:42913)
0807 22:35:44.922815 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:44.922820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:44.922832 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:35:44.922873 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:44.922875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:44.922891 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:44.922983 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:44.923219 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.923925 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.925048 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.925788 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.925868 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.925920 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:44.926094 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 22:35:44.926132 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260807 22:35:44.926211 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.922369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:42913 (local address 127.0.0.1:38200)
0807 22:35:44.922749 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:44.922759 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:44.922801 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:44.923066 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:44.923067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:44.923069 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:44.923142 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.923143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.924030 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.924032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.924951 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.924954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.925885 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.925886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.925968 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.925982 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:35:44.926065 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:35:44.926128 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":27,"thread_start_us":88,"threads_started":1}
I20260807 22:35:44.926430 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.922345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:40685 (local address 127.0.0.1:51322)
0807 22:35:44.922555 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:44.922597 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:44.922671 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:44.922900 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:44.922903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:44.922905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:44.922968 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.922971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.923980 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.923981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.925386 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.925389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.926187 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.926188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.926261 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.926264 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:44.926317 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 22:35:44.926363 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":43,"thread_start_us":59,"threads_started":1}
I20260807 22:35:44.926721 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.922453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.28.82.193:40685)
0807 22:35:44.922678 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:44.922683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:44.922695 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:35:44.922793 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:44.922796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:44.922814 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:44.922897 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:44.923046 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.923870 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.925454 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.926023 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.926086 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.926140 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:44.926599 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 22:35:44.926644 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":97,"threads_started":1}
I20260807 22:35:44.927366 2594 tablet_service.cc:1467] Processing CreateTablet for tablet 836a926dcc9c491a8e17ac86b2a412e9 (DEFAULT_TABLE table=test-table [id=83a409bf807f47179d4be2fd4799ef40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:44.927371 2961 tablet_service.cc:1467] Processing CreateTablet for tablet 836a926dcc9c491a8e17ac86b2a412e9 (DEFAULT_TABLE table=test-table [id=83a409bf807f47179d4be2fd4799ef40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:44.927525 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.922368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:44619 (local address 127.0.0.1:40358)
0807 22:35:44.922955 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:44.922966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:44.923026 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:44.924147 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:44.924148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:44.924152 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:44.924190 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.924192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.925381 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.925382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.926427 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:44.926430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.927293 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.927294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:44.927356 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.927361 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:35:44.927419 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 22:35:44.927457 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":3,"thread_start_us":130,"threads_started":1}
I20260807 22:35:44.927547 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:44.922695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40358 (local address 127.28.82.194:44619)
0807 22:35:44.922899 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:44.922903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:44.923992 (+ 1089us) server_negotiation.cc:407] Connection header received
0807 22:35:44.924040 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:44.924043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:44.924060 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:44.924138 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:44.924256 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.925296 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.926533 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:44.927190 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:44.927292 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:44.927344 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:44.927433 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 22:35:44.927471 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260807 22:35:44.927750 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836a926dcc9c491a8e17ac86b2a412e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:44.927745 2594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836a926dcc9c491a8e17ac86b2a412e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:44.928112 2775 tablet_service.cc:1467] Processing CreateTablet for tablet 836a926dcc9c491a8e17ac86b2a412e9 (DEFAULT_TABLE table=test-table [id=83a409bf807f47179d4be2fd4799ef40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:44.928371 2775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836a926dcc9c491a8e17ac86b2a412e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:44.931005 3083 tablet_bootstrap.cc:492] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Bootstrap starting.
I20260807 22:35:44.931542 3085 tablet_bootstrap.cc:492] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0: Bootstrap starting.
I20260807 22:35:44.931584 3083 tablet_bootstrap.cc:654] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:44.932124 3085 tablet_bootstrap.cc:654] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:44.932125 3083 log.cc:824] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:44.932760 3085 log.cc:824] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:44.933022 3083 tablet_bootstrap.cc:492] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: No bootstrap required, opened a new log
I20260807 22:35:44.933104 3083 ts_tablet_manager.cc:1397] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:44.933122 3084 tablet_bootstrap.cc:492] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: Bootstrap starting.
I20260807 22:35:44.933506 3085 tablet_bootstrap.cc:492] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0: No bootstrap required, opened a new log
I20260807 22:35:44.933631 3085 ts_tablet_manager.cc:1397] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:35:44.933835 3084 tablet_bootstrap.cc:654] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:44.934537 3084 log.cc:824] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:44.935359 3084 tablet_bootstrap.cc:492] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: No bootstrap required, opened a new log
I20260807 22:35:44.935443 3084 ts_tablet_manager.cc:1397] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 22:35:44.935559 3083 raft_consensus.cc:348] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:44.935737 3083 raft_consensus.cc:374] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:44.935775 3083 raft_consensus.cc:729] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26be0da92c2648c99941411b9a961982, State: Initialized, Role: FOLLOWER
I20260807 22:35:44.935892 3083 consensus_queue.cc:260] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:44.935887 3085 raft_consensus.cc:348] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:44.936035 3085 raft_consensus.cc:374] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:44.936082 3085 raft_consensus.cc:729] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f287c578a409494eaf1e70ce1a5d42f0, State: Initialized, Role: FOLLOWER
I20260807 22:35:44.936162 3083 ts_tablet_manager.cc:1428] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:44.936213 3085 consensus_queue.cc:260] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:44.936198 2868 heartbeater.cc:503] Master 127.28.82.254:39567 was elected leader, sending a full tablet report...
I20260807 22:35:44.936508 3085 ts_tablet_manager.cc:1428] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 22:35:44.936537 2676 heartbeater.cc:503] Master 127.28.82.254:39567 was elected leader, sending a full tablet report...
I20260807 22:35:44.937495 3084 raft_consensus.cc:348] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:44.937645 3084 raft_consensus.cc:374] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:44.937670 3084 raft_consensus.cc:729] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abef618bd2b049ebb1b2107dcc5a9148, State: Initialized, Role: FOLLOWER
I20260807 22:35:44.937796 3084 consensus_queue.cc:260] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:44.938040 3084 ts_tablet_manager.cc:1428] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 22:35:44.938076 3060 heartbeater.cc:503] Master 127.28.82.254:39567 was elected leader, sending a full tablet report...
W20260807 22:35:44.978098 2872 tablet.cc:2369] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:45.045120 3091 raft_consensus.cc:482] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:45.045264 3091 raft_consensus.cc:504] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:45.045723 3091 leader_election.cc:288] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abef618bd2b049ebb1b2107dcc5a9148 (127.28.82.195:42913), 26be0da92c2648c99941411b9a961982 (127.28.82.194:44619)
I20260807 22:35:45.049068 3080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.046080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:57077 (local address 127.28.82.195:42913)
0807 22:35:45.046142 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.046145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.046155 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:45.046187 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.046188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.046200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.046263 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:45.046499 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.047110 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.048232 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.048702 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.048785 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:45.048811 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.048963 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.048997 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:35:45.049068 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.045965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:42913 (local address 127.28.82.193:57077)
0807 22:35:45.046032 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.046045 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.046090 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.046290 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.046292 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.046294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:45.046371 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:45.046373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.047179 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.047180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:45.048140 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:45.048143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.048783 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.048784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:45.048878 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:45.048882 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.048946 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.048988 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 22:35:45.049176 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.045966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:44619 (local address 127.28.82.193:40429)
0807 22:35:45.046168 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.046179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.046254 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.046472 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.046474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.046477 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:45.046616 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:45.046619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.047438 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.047439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:45.048251 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:45.048254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.048887 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.048888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:45.048970 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:45.048974 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.049049 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.049092 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":16,"thread_start_us":87,"threads_started":1}
I20260807 22:35:45.049165 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.046081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:40429 (local address 127.28.82.194:44619)
0807 22:35:45.046147 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.046150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.046274 (+ 124us) server_negotiation.cc:407] Connection header received
0807 22:35:45.046306 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.046307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.046330 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.046397 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:45.046726 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.047352 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.048352 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.048798 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.048887 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:45.048896 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.049058 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.049095 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:35:45.049474 2998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836a926dcc9c491a8e17ac86b2a412e9" candidate_uuid: "f287c578a409494eaf1e70ce1a5d42f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abef618bd2b049ebb1b2107dcc5a9148" is_pre_election: true
I20260807 22:35:45.049610 2998 raft_consensus.cc:2454] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f287c578a409494eaf1e70ce1a5d42f0 in term 0.
I20260807 22:35:45.049885 2527 leader_election.cc:302] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abef618bd2b049ebb1b2107dcc5a9148, f287c578a409494eaf1e70ce1a5d42f0; no voters:
I20260807 22:35:45.050048 3091 raft_consensus.cc:2781] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:45.050040 2803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836a926dcc9c491a8e17ac86b2a412e9" candidate_uuid: "f287c578a409494eaf1e70ce1a5d42f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26be0da92c2648c99941411b9a961982" is_pre_election: true
I20260807 22:35:45.050122 3091 raft_consensus.cc:482] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:45.050153 3091 raft_consensus.cc:3037] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:45.050182 2803 raft_consensus.cc:2454] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f287c578a409494eaf1e70ce1a5d42f0 in term 0.
I20260807 22:35:45.050944 3091 raft_consensus.cc:504] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:45.051110 3091 leader_election.cc:288] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [CANDIDATE]: Term 1 election: Requested vote from peers abef618bd2b049ebb1b2107dcc5a9148 (127.28.82.195:42913), 26be0da92c2648c99941411b9a961982 (127.28.82.194:44619)
I20260807 22:35:45.051285 2803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836a926dcc9c491a8e17ac86b2a412e9" candidate_uuid: "f287c578a409494eaf1e70ce1a5d42f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26be0da92c2648c99941411b9a961982"
I20260807 22:35:45.051378 2998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836a926dcc9c491a8e17ac86b2a412e9" candidate_uuid: "f287c578a409494eaf1e70ce1a5d42f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abef618bd2b049ebb1b2107dcc5a9148"
I20260807 22:35:45.051375 2803 raft_consensus.cc:3037] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:45.051476 2998 raft_consensus.cc:3037] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:45.052145 2803 raft_consensus.cc:2454] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f287c578a409494eaf1e70ce1a5d42f0 in term 1.
I20260807 22:35:45.052270 2998 raft_consensus.cc:2454] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f287c578a409494eaf1e70ce1a5d42f0 in term 1.
I20260807 22:35:45.052326 2532 leader_election.cc:302] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26be0da92c2648c99941411b9a961982, f287c578a409494eaf1e70ce1a5d42f0; no voters:
I20260807 22:35:45.052449 3091 raft_consensus.cc:2781] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:45.052662 3091 raft_consensus.cc:686] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [term 1 LEADER]: Becoming Leader. State: Replica: f287c578a409494eaf1e70ce1a5d42f0, State: Running, Role: LEADER
I20260807 22:35:45.052799 3091 consensus_queue.cc:237] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } }
I20260807 22:35:45.053795 2420 catalog_manager.cc:5495] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f287c578a409494eaf1e70ce1a5d42f0 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "f287c578a409494eaf1e70ce1a5d42f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f287c578a409494eaf1e70ce1a5d42f0" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 } health_report { overall_health: UNKNOWN } } }
W20260807 22:35:45.067654 2677 tablet.cc:2369] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:45.075976 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.072997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.28.82.193:40685)
0807 22:35:45.073089 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.073091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.073101 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:45.073135 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.073136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.073155 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.073237 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:45.073398 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.073948 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.074874 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:45.075127 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:45.075188 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:45.075443 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:45.075596 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:45.075683 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.075798 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.075851 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:35:45.077980 2998 raft_consensus.cc:1261] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 1 FOLLOWER]: Refusing update from remote peer f287c578a409494eaf1e70ce1a5d42f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:45.078006 2803 raft_consensus.cc:1261] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 1 FOLLOWER]: Refusing update from remote peer f287c578a409494eaf1e70ce1a5d42f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:45.078352 3091 consensus_queue.cc:1035] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26be0da92c2648c99941411b9a961982" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 44619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:45.078537 3095 consensus_queue.cc:1035] T 836a926dcc9c491a8e17ac86b2a412e9 P f287c578a409494eaf1e70ce1a5d42f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abef618bd2b049ebb1b2107dcc5a9148" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:45.080754 3099 mvcc.cc:205] Tried to move back new op lower bound from 7316038226236669952 to 7316038226137010176. Current Snapshot: MvccSnapshot[applied={T|T < 7316038226236669952}]
I20260807 22:35:45.083768 2419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:45.083856 2419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:45.084676 2419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18808635d1914cc78722000cd037d68f: Sending DeleteTablet for 3 replicas of tablet 836a926dcc9c491a8e17ac86b2a412e9
I20260807 22:35:45.085004 2594 tablet_service.cc:1514] Processing DeleteTablet for tablet 836a926dcc9c491a8e17ac86b2a412e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:45 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:51322
I20260807 22:35:45.085039 2775 tablet_service.cc:1514] Processing DeleteTablet for tablet 836a926dcc9c491a8e17ac86b2a412e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:45 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40358
I20260807 22:35:45.085246 2961 tablet_service.cc:1514] Processing DeleteTablet for tablet 836a926dcc9c491a8e17ac86b2a412e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:45 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:38200
I20260807 22:35:45.085348 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 2502
I20260807 22:35:45.085467 3114 tablet_replica.cc:330] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: stopping tablet replica
I20260807 22:35:45.085589 3114 raft_consensus.cc:2229] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:45.085711 3114 raft_consensus.cc:2258] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:45.085958 3113 tablet_replica.cc:330] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: stopping tablet replica
I20260807 22:35:45.086066 3113 raft_consensus.cc:2229] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:45.086169 3113 raft_consensus.cc:2258] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:45.086648 3113 ts_tablet_manager.cc:1905] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:45.088497 3113 ts_tablet_manager.cc:1918] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:45.088636 3113 log.cc:1196] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/836a926dcc9c491a8e17ac86b2a412e9
I20260807 22:35:45.088980 3113 ts_tablet_manager.cc:1939] T 836a926dcc9c491a8e17ac86b2a412e9 P 26be0da92c2648c99941411b9a961982: Deleting consensus metadata
I20260807 22:35:45.090538 2403 catalog_manager.cc:4841] TS 26be0da92c2648c99941411b9a961982 (127.28.82.194:44619): tablet 836a926dcc9c491a8e17ac86b2a412e9 (table test-table [id=83a409bf807f47179d4be2fd4799ef40]) successfully deleted
I20260807 22:35:45.090751 3114 ts_tablet_manager.cc:1905] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:45.092989 3114 ts_tablet_manager.cc:1918] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:45.093077 3114 log.cc:1196] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/836a926dcc9c491a8e17ac86b2a412e9
I20260807 22:35:45.093390 3114 ts_tablet_manager.cc:1939] T 836a926dcc9c491a8e17ac86b2a412e9 P abef618bd2b049ebb1b2107dcc5a9148: Deleting consensus metadata
I20260807 22:35:45.093854 2399 catalog_manager.cc:4841] TS abef618bd2b049ebb1b2107dcc5a9148 (127.28.82.195:42913): tablet 836a926dcc9c491a8e17ac86b2a412e9 (table test-table [id=83a409bf807f47179d4be2fd4799ef40]) successfully deleted
W20260807 22:35:45.097654 2402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:40685 (error 108)
I20260807 22:35:45.098028 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 2682
I20260807 22:35:45.098250 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.098098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:40685 (local address 127.0.0.1:51338)
0807 22:35:45.098163 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.098208 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:40685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 22:35:45.107983 2402 catalog_manager.cc:4584] TS f287c578a409494eaf1e70ce1a5d42f0 (127.28.82.193:40685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 836a926dcc9c491a8e17ac86b2a412e9: Network error: Client connection negotiation failed: client connection to 127.28.82.193:40685: connect: Connection refused (error 111)
I20260807 22:35:45.112869 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 2878
I20260807 22:35:45.121114 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 2377
2026-08-07T22:35:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2218 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T22:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:45Z Disabled control of system clock
I20260807 22:35:45.157037 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43845
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:39835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:43845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:35:45.240352 3123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:45.240556 3123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:45.240577 3123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:45.241779 3123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:45.241819 3123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:45.241832 3123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:45.241845 3123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:45.241891 3123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:45.243299 3123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:39835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:43845
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:45.243551 3123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:45.243880 3123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:45.248872 3133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:45.248862 3137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:45.248864 3131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:45.249087 3123 server_base.cc:1034] running on GCE node
I20260807 22:35:45.249403 3123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:45.249691 3123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:45.250855 3123 hybrid_clock.cc:648] HybridClock initialized: now 1786142145250849 us; error 36 us; skew 500 ppm
I20260807 22:35:45.252113 3123 webserver.cc:460] Webserver started at http://127.28.82.254:36865/ using document root <none> and password file <none>
I20260807 22:35:45.252357 3123 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:45.252425 3123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:45.252583 3123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:45.253631 3123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "477f1bea7cf841178cfcb5970a09d5ca"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "9722c5e0fc00f2c3eb00c4cc2a7517a4"
server_key_iv: "c56000866bd68ccd020a41d1afa364ca"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.254004 3123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "477f1bea7cf841178cfcb5970a09d5ca"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "9722c5e0fc00f2c3eb00c4cc2a7517a4"
server_key_iv: "c56000866bd68ccd020a41d1afa364ca"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.255384 3123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:45.256168 3148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.256347 3123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:45.256456 3123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "477f1bea7cf841178cfcb5970a09d5ca"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "9722c5e0fc00f2c3eb00c4cc2a7517a4"
server_key_iv: "c56000866bd68ccd020a41d1afa364ca"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.256531 3123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:45.259882 3123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:45.260151 3123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:45.260296 3123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:45.264437 3123 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:43845
I20260807 22:35:45.264467 3226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:43845 every 8 connection(s)
I20260807 22:35:45.264806 3123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:45.265450 3227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:45.268193 3227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca: Bootstrap starting.
I20260807 22:35:45.268750 3227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:45.269054 3227 log.cc:824] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:45.269774 3227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca: No bootstrap required, opened a new log
I20260807 22:35:45.271526 3227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477f1bea7cf841178cfcb5970a09d5ca" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43845 } }
I20260807 22:35:45.271664 3227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:45.271687 3227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477f1bea7cf841178cfcb5970a09d5ca, State: Initialized, Role: FOLLOWER
I20260807 22:35:45.271837 3227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477f1bea7cf841178cfcb5970a09d5ca" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43845 } }
I20260807 22:35:45.271921 3227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:45.271975 3227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:45.272034 3227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:45.272130 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 3123
I20260807 22:35:45.272240 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:45.272428 29003 external_mini_cluster.cc:1428] Setting key bd08efcad62ad8e9c12aeee6005f3d8e
I20260807 22:35:45.272789 3227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477f1bea7cf841178cfcb5970a09d5ca" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43845 } }
I20260807 22:35:45.272921 3227 leader_election.cc:302] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 477f1bea7cf841178cfcb5970a09d5ca; no voters:
I20260807 22:35:45.273108 3227 leader_election.cc:288] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:45.273145 3234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:45.273315 3234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [term 1 LEADER]: Becoming Leader. State: Replica: 477f1bea7cf841178cfcb5970a09d5ca, State: Running, Role: LEADER
I20260807 22:35:45.273373 3227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:45.273442 3234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477f1bea7cf841178cfcb5970a09d5ca" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43845 } }
I20260807 22:35:45.273761 3234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 477f1bea7cf841178cfcb5970a09d5ca. Latest consensus state: current_term: 1 leader_uuid: "477f1bea7cf841178cfcb5970a09d5ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477f1bea7cf841178cfcb5970a09d5ca" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43845 } } }
I20260807 22:35:45.273921 3234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:45.273979 3236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "477f1bea7cf841178cfcb5970a09d5ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "477f1bea7cf841178cfcb5970a09d5ca" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43845 } } }
I20260807 22:35:45.274086 3236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:45.274948 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.272924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.28.82.254:43845)
0807 22:35:45.273267 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.273277 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.273306 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:35:45.273416 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.273424 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.273470 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.273889 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.274460 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.274490 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.274491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.274495 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.274541 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.274557 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.274620 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.274701 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.274765 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":88,"threads_started":1}
I20260807 22:35:45.275010 3238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:45.275548 3238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:45.277201 3238 catalog_manager.cc:1347] Generated new cluster ID: 3885bf095ca1497fa3664ef45b7b49eb
I20260807 22:35:45.277261 3238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:45.285401 3238 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:45.285552 3238 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:45.289714 3238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca: Generated new TSK 0
I20260807 22:35:45.289880 3238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:35:45.294287 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43845
--builtin_ntp_servers=127.28.82.212:39835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:35:45.382365 3259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:45.382627 3259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:45.382668 3259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:45.383823 3259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:45.383890 3259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:45.383935 3259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:45.385319 3259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:39835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:45.385588 3259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:45.385895 3259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:45.391840 3266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:45.391937 3272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:45.391877 3267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:45.392268 3259 server_base.cc:1034] running on GCE node
I20260807 22:35:45.392490 3259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:45.392747 3259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:45.393913 3259 hybrid_clock.cc:648] HybridClock initialized: now 1786142145393885 us; error 44 us; skew 500 ppm
I20260807 22:35:45.395292 3259 webserver.cc:460] Webserver started at http://127.28.82.193:42749/ using document root <none> and password file <none>
I20260807 22:35:45.395560 3259 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:45.395632 3259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:45.395825 3259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:45.396907 3259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "8f480476145246eca7f22b87f9409295"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "1ad09e51dbc958de2c41a8504cafba06"
server_key_iv: "d6635409d42e2a69a85647edb6b9c7e9"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.397276 3259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "8f480476145246eca7f22b87f9409295"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "1ad09e51dbc958de2c41a8504cafba06"
server_key_iv: "d6635409d42e2a69a85647edb6b9c7e9"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.398727 3259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:45.399487 3283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.399722 3259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:45.399827 3259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "8f480476145246eca7f22b87f9409295"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "1ad09e51dbc958de2c41a8504cafba06"
server_key_iv: "d6635409d42e2a69a85647edb6b9c7e9"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.399920 3259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:45.403322 3259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:45.403609 3259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:45.403762 3259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:45.403976 3259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:45.404292 3259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:45.404347 3259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.404388 3259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:45.404415 3259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.410143 3259 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:43521
I20260807 22:35:45.410226 3456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:43521 every 8 connection(s)
I20260807 22:35:45.410634 3259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:45.414031 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.411497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:55897 (local address 127.28.82.254:43845)
0807 22:35:45.411611 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.411613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.411827 (+ 214us) server_negotiation.cc:407] Connection header received
0807 22:35:45.412367 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.412369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.412377 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.412434 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.413709 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.413711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.413711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.413716 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.413734 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.413739 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.413811 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.413932 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.413957 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 22:35:45.414269 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.411404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43845 (local address 127.28.82.193:55897)
0807 22:35:45.411691 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.411740 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.412255 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.412519 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.412525 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.412539 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:45.413540 (+ 1001us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:35:45.413541 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:45.413573 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:35:45.413601 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:45.413810 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:45.413818 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:45.413821 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.413937 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.414014 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":115,"threads_started":1}
I20260807 22:35:45.414871 3457 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43845
I20260807 22:35:45.414953 3457 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:45.415112 3457 heartbeater.cc:511] Master 127.28.82.254:43845 requested a full tablet report, sending...
I20260807 22:35:45.415760 3167 ts_manager.cc:194] Registered new tserver with Master: 8f480476145246eca7f22b87f9409295 (127.28.82.193:43521)
I20260807 22:35:45.420022 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 3259
I20260807 22:35:45.420158 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:45.420358 29003 external_mini_cluster.cc:1428] Setting key 30fab47bf1e372f4066b827a6685902c
I20260807 22:35:45.421380 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43845
--builtin_ntp_servers=127.28.82.212:39835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 22:35:45.506361 3462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:45.506667 3462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:45.506707 3462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:45.507893 3462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:45.507964 3462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:45.508008 3462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:45.509599 3462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:39835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:45.509888 3462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:45.510201 3462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:45.515684 3475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:45.515646 3470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:45.515641 3469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:45.515929 3462 server_base.cc:1034] running on GCE node
I20260807 22:35:45.516163 3462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:45.516419 3462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:45.517570 3462 hybrid_clock.cc:648] HybridClock initialized: now 1786142145517557 us; error 31 us; skew 500 ppm
I20260807 22:35:45.518998 3462 webserver.cc:460] Webserver started at http://127.28.82.194:43359/ using document root <none> and password file <none>
I20260807 22:35:45.519248 3462 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:45.519322 3462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:45.519464 3462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:45.520525 3462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "4dcd7527d63a4406804308b74fe6e738"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "8a30b963ba691d0d9348238adf99c88e"
server_key_iv: "255758e8642a44e07454030afcbc3082"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.520910 3462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "4dcd7527d63a4406804308b74fe6e738"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "8a30b963ba691d0d9348238adf99c88e"
server_key_iv: "255758e8642a44e07454030afcbc3082"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.522243 3462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:35:45.523068 3485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.523289 3462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:45.523406 3462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "4dcd7527d63a4406804308b74fe6e738"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "8a30b963ba691d0d9348238adf99c88e"
server_key_iv: "255758e8642a44e07454030afcbc3082"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.523505 3462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:45.527009 3462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:45.527293 3462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:45.527456 3462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:45.527707 3462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:45.528016 3462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:45.528077 3462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.528119 3462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:45.528146 3462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.533852 3462 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:46857
I20260807 22:35:45.533936 3653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:46857 every 8 connection(s)
I20260807 22:35:45.534262 3462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:45.537168 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 3462
I20260807 22:35:45.537272 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:45.537488 29003 external_mini_cluster.cc:1428] Setting key a01a934990433727b96209a0f5b3e2a4
I20260807 22:35:45.537501 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.535087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:32973 (local address 127.28.82.254:43845)
0807 22:35:45.535155 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.535157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.535412 (+ 255us) server_negotiation.cc:407] Connection header received
0807 22:35:45.535852 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.535853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.535857 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.535918 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.537144 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.537147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.537148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.537154 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.537175 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.537181 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.537256 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.537401 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.537422 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:35:45.537658 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.535075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43845 (local address 127.28.82.194:32973)
0807 22:35:45.535267 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.535308 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.535766 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.535967 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.535970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.535978 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:45.536976 (+ 998us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:35:45.536976 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:45.537004 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:35:45.537034 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:45.537309 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:45.537315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:45.537318 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.537400 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.537450 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260807 22:35:45.538201 3654 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43845
I20260807 22:35:45.538324 3654 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:45.538513 3654 heartbeater.cc:511] Master 127.28.82.254:43845 requested a full tablet report, sending...
I20260807 22:35:45.538630 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43845
--builtin_ntp_servers=127.28.82.212:39835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 22:35:45.538893 3167 ts_manager.cc:194] Registered new tserver with Master: 4dcd7527d63a4406804308b74fe6e738 (127.28.82.194:46857)
W20260807 22:35:45.625279 3657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:45.625526 3657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:45.625562 3657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:45.626791 3657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:45.626854 3657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:45.626897 3657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:45.628296 3657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:39835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:45.628556 3657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:45.628855 3657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:45.633814 3663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:45.633775 3664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:45.633769 3668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:45.634128 3657 server_base.cc:1034] running on GCE node
I20260807 22:35:45.634341 3657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:45.634614 3657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:45.635815 3657 hybrid_clock.cc:648] HybridClock initialized: now 1786142145635800 us; error 30 us; skew 500 ppm
I20260807 22:35:45.637161 3657 webserver.cc:460] Webserver started at http://127.28.82.195:43729/ using document root <none> and password file <none>
I20260807 22:35:45.637434 3657 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:45.637485 3657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:45.637627 3657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:45.638782 3657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "743e793e6dd9451bac7bba45957f185a"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "536a5951655d2c5329ea30bb88214e65"
server_key_iv: "3fefb8a2ee0fc05fa0cd544b1acde857"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.639146 3657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "743e793e6dd9451bac7bba45957f185a"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "536a5951655d2c5329ea30bb88214e65"
server_key_iv: "3fefb8a2ee0fc05fa0cd544b1acde857"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.640514 3657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:45.641326 3675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.641546 3657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:45.641655 3657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "743e793e6dd9451bac7bba45957f185a"
format_stamp: "Formatted at 2026-08-07 22:35:45 on dist-test-slave-7kzw"
server_key: "536a5951655d2c5329ea30bb88214e65"
server_key_iv: "3fefb8a2ee0fc05fa0cd544b1acde857"
server_key_version: "encryptionkey@0"
I20260807 22:35:45.641731 3657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:45.645167 3657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:45.645462 3657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:45.645605 3657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:45.645840 3657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:45.646152 3657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:45.646186 3657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.646238 3657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:45.646267 3657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:45.652333 3657 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:35679
I20260807 22:35:45.652412 3839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:35679 every 8 connection(s)
I20260807 22:35:45.652755 3657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:45.654641 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 3657
I20260807 22:35:45.654778 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:45.654959 29003 external_mini_cluster.cc:1428] Setting key 7940737b4f77067903c01a91a20b644f
I20260807 22:35:45.655993 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.653532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:53005 (local address 127.28.82.254:43845)
0807 22:35:45.653623 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.653626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.653827 (+ 201us) server_negotiation.cc:407] Connection header received
0807 22:35:45.654250 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.654251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.654257 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.654311 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.655758 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.655759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.655760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.655763 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.655778 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.655781 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.655821 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.655897 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.655912 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:35:45.656195 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.653489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43845 (local address 127.28.82.195:53005)
0807 22:35:45.653710 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.653742 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.654168 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.654362 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.654365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.654372 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:45.655589 (+ 1217us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:35:45.655590 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:45.655637 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:35:45.655682 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:45.655832 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:45.655836 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:45.655839 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.655910 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.655956 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
I20260807 22:35:45.656644 3840 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43845
I20260807 22:35:45.656718 3840 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:45.656868 3840 heartbeater.cc:511] Master 127.28.82.254:43845 requested a full tablet report, sending...
I20260807 22:35:45.657196 3167 ts_manager.cc:194] Registered new tserver with Master: 743e793e6dd9451bac7bba45957f185a (127.28.82.195:35679)
I20260807 22:35:45.666972 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:35:45.669816 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.668572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59460 (local address 127.28.82.254:43845)
0807 22:35:45.668642 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.668644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.668775 (+ 131us) server_negotiation.cc:407] Connection header received
0807 22:35:45.668855 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.668856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.668861 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.668917 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.669581 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.669582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.669583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.669587 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.669604 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.669609 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.669669 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.669725 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.669741 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:35:45.671548 3167 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:45.673043 3167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:45.676891 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.675234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43521 (local address 127.0.0.1:45602)
0807 22:35:45.675534 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.675574 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.675655 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.675884 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.675889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.675898 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:45.676411 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:35:45.676411 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:45.676428 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:35:45.676450 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:45.676696 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:45.676703 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:45.676706 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.676805 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.676819 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":63,"threads_started":1}
I20260807 22:35:45.676941 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.675325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45602 (local address 127.28.82.193:43521)
0807 22:35:45.675758 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.675762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.675772 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:45.675797 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.675799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.675809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.675885 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.676551 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.676560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.676561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.676566 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.676600 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.676629 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.676689 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.676815 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.676832 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":99,"threads_started":1}
I20260807 22:35:45.677266 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.675256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:46857 (local address 127.0.0.1:39486)
0807 22:35:45.675583 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.675595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.675695 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.676473 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.676474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.676477 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:45.676866 (+ 389us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:35:45.676866 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:45.676873 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:35:45.676884 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:45.677133 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:45.677134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:45.677134 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.677186 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.677198 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":84,"threads_started":1}
I20260807 22:35:45.677549 3854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.675361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.28.82.194:46857)
0807 22:35:45.676321 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.676325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.676333 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:45.676364 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.676366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.676372 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.676458 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.676971 (+ 513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.676981 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.676981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.676986 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.677025 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.677057 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.677109 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.677448 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.677466 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":105,"threads_started":1}
I20260807 22:35:45.677644 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.675441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:35679 (local address 127.0.0.1:59822)
0807 22:35:45.675743 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.675752 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.675806 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.676388 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.676389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.676393 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:45.677272 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:35:45.677273 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:45.677280 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:35:45.677291 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:45.677511 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:45.677512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:45.677512 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.677554 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.677567 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":68,"threads_started":1}
I20260807 22:35:45.677644 3351 tablet_service.cc:1467] Processing CreateTablet for tablet 41960e4a58d149ac9ea545ed4b6d74ce (DEFAULT_TABLE table=test-table [id=d306bfd676e349498dbcf1cab9180e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:45.677881 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.675521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.28.82.195:35679)
0807 22:35:45.675717 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.675722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.675802 (+ 80us) server_negotiation.cc:407] Connection header received
0807 22:35:45.675858 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.675861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.675870 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.675938 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.677372 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.677377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.677377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.677381 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.677409 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.677427 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.677477 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.677783 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.677803 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260807 22:35:45.678046 3351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41960e4a58d149ac9ea545ed4b6d74ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:45.678174 3556 tablet_service.cc:1467] Processing CreateTablet for tablet 41960e4a58d149ac9ea545ed4b6d74ce (DEFAULT_TABLE table=test-table [id=d306bfd676e349498dbcf1cab9180e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:45.678457 3556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41960e4a58d149ac9ea545ed4b6d74ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:45.678432 3743 tablet_service.cc:1467] Processing CreateTablet for tablet 41960e4a58d149ac9ea545ed4b6d74ce (DEFAULT_TABLE table=test-table [id=d306bfd676e349498dbcf1cab9180e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:45.678787 3743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41960e4a58d149ac9ea545ed4b6d74ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:45.681154 3861 tablet_bootstrap.cc:492] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Bootstrap starting.
I20260807 22:35:45.681748 3861 tablet_bootstrap.cc:654] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:45.681957 3862 tablet_bootstrap.cc:492] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295: Bootstrap starting.
I20260807 22:35:45.682090 3863 tablet_bootstrap.cc:492] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: Bootstrap starting.
I20260807 22:35:45.682129 3861 log.cc:824] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:45.682813 3862 tablet_bootstrap.cc:654] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:45.682857 3863 tablet_bootstrap.cc:654] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:45.683060 3861 tablet_bootstrap.cc:492] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: No bootstrap required, opened a new log
I20260807 22:35:45.683177 3863 log.cc:824] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:45.683188 3862 log.cc:824] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:45.683257 3861 ts_tablet_manager.cc:1397] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:35:45.683885 3862 tablet_bootstrap.cc:492] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295: No bootstrap required, opened a new log
I20260807 22:35:45.683971 3862 ts_tablet_manager.cc:1397] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:45.685194 3861 raft_consensus.cc:348] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.685322 3861 raft_consensus.cc:374] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:45.685344 3861 raft_consensus.cc:729] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dcd7527d63a4406804308b74fe6e738, State: Initialized, Role: FOLLOWER
I20260807 22:35:45.685472 3861 consensus_queue.cc:260] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.685763 3861 ts_tablet_manager.cc:1428] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:35:45.685792 3654 heartbeater.cc:503] Master 127.28.82.254:43845 was elected leader, sending a full tablet report...
I20260807 22:35:45.685945 3862 raft_consensus.cc:348] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.686121 3862 raft_consensus.cc:374] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:45.686179 3862 raft_consensus.cc:729] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f480476145246eca7f22b87f9409295, State: Initialized, Role: FOLLOWER
I20260807 22:35:45.686305 3862 consensus_queue.cc:260] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.686589 3862 ts_tablet_manager.cc:1428] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:45.686707 3457 heartbeater.cc:503] Master 127.28.82.254:43845 was elected leader, sending a full tablet report...
I20260807 22:35:45.687332 3863 tablet_bootstrap.cc:492] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: No bootstrap required, opened a new log
I20260807 22:35:45.687407 3863 ts_tablet_manager.cc:1397] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260807 22:35:45.689242 3863 raft_consensus.cc:348] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.689360 3863 raft_consensus.cc:374] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:45.689381 3863 raft_consensus.cc:729] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743e793e6dd9451bac7bba45957f185a, State: Initialized, Role: FOLLOWER
I20260807 22:35:45.689484 3863 consensus_queue.cc:260] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.689744 3863 ts_tablet_manager.cc:1428] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 22:35:45.689775 3840 heartbeater.cc:503] Master 127.28.82.254:43845 was elected leader, sending a full tablet report...
W20260807 22:35:45.784870 3655 tablet.cc:2369] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:45.843242 3873 raft_consensus.cc:482] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:45.843415 3873 raft_consensus.cc:504] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.843864 3873 leader_election.cc:288] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f480476145246eca7f22b87f9409295 (127.28.82.193:43521), 4dcd7527d63a4406804308b74fe6e738 (127.28.82.194:46857)
I20260807 22:35:45.845382 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.844112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:46857 (local address 127.28.82.195:40257)
0807 22:35:45.844168 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.844182 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.844229 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.844472 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.844473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.844476 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:45.845027 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:35:45.845028 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:45.845037 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:35:45.845053 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:45.845254 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:45.845255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:45.845256 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.845302 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.845317 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 22:35:45.845634 3854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.844274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:40257 (local address 127.28.82.194:46857)
0807 22:35:45.844336 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.844338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.844345 (+ 7us) server_negotiation.cc:407] Connection header received
0807 22:35:45.844368 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.844369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.844374 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.844431 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.845144 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.845145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.845146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.845151 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.845167 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.845174 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.845227 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.845516 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.845534 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 22:35:45.845722 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.844207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:42195 (local address 127.28.82.193:43521)
0807 22:35:45.844279 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.844281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.844610 (+ 329us) server_negotiation.cc:407] Connection header received
0807 22:35:45.844641 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.844642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.844649 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.844695 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.845371 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.845372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.845372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.845378 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.845395 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.845400 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.845459 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.845637 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.845654 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:35:45.845903 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.844136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43521 (local address 127.28.82.195:42195)
0807 22:35:45.844474 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.844486 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:45.844550 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:45.844712 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:45.844713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:45.844716 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:45.845159 (+ 443us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:35:45.845159 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:45.845167 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:35:45.845204 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:45.845536 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:45.845537 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:45.845538 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 22:35:45.845808 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 22:35:45.845824 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":106,"thread_start_us":152,"threads_started":1}
I20260807 22:35:45.846046 3584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41960e4a58d149ac9ea545ed4b6d74ce" candidate_uuid: "743e793e6dd9451bac7bba45957f185a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dcd7527d63a4406804308b74fe6e738" is_pre_election: true
I20260807 22:35:45.846212 3584 raft_consensus.cc:2454] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 743e793e6dd9451bac7bba45957f185a in term 0.
I20260807 22:35:45.846262 3386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41960e4a58d149ac9ea545ed4b6d74ce" candidate_uuid: "743e793e6dd9451bac7bba45957f185a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f480476145246eca7f22b87f9409295" is_pre_election: true
I20260807 22:35:45.846410 3386 raft_consensus.cc:2454] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 743e793e6dd9451bac7bba45957f185a in term 0.
I20260807 22:35:45.846691 3681 leader_election.cc:302] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dcd7527d63a4406804308b74fe6e738, 743e793e6dd9451bac7bba45957f185a; no voters:
I20260807 22:35:45.846819 3873 raft_consensus.cc:2781] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:45.846879 3873 raft_consensus.cc:482] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:45.846910 3873 raft_consensus.cc:3037] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:45.848022 3873 raft_consensus.cc:504] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.848170 3873 leader_election.cc:288] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [CANDIDATE]: Term 1 election: Requested vote from peers 8f480476145246eca7f22b87f9409295 (127.28.82.193:43521), 4dcd7527d63a4406804308b74fe6e738 (127.28.82.194:46857)
I20260807 22:35:45.848352 3386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41960e4a58d149ac9ea545ed4b6d74ce" candidate_uuid: "743e793e6dd9451bac7bba45957f185a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f480476145246eca7f22b87f9409295"
I20260807 22:35:45.848444 3386 raft_consensus.cc:3037] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:45.848472 3584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41960e4a58d149ac9ea545ed4b6d74ce" candidate_uuid: "743e793e6dd9451bac7bba45957f185a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dcd7527d63a4406804308b74fe6e738"
I20260807 22:35:45.848536 3584 raft_consensus.cc:3037] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:45.849383 3584 raft_consensus.cc:2454] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 743e793e6dd9451bac7bba45957f185a in term 1.
I20260807 22:35:45.849385 3386 raft_consensus.cc:2454] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 743e793e6dd9451bac7bba45957f185a in term 1.
I20260807 22:35:45.849558 3677 leader_election.cc:302] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 743e793e6dd9451bac7bba45957f185a, 8f480476145246eca7f22b87f9409295; no voters:
I20260807 22:35:45.849663 3873 raft_consensus.cc:2781] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:45.849817 3873 raft_consensus.cc:686] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 1 LEADER]: Becoming Leader. State: Replica: 743e793e6dd9451bac7bba45957f185a, State: Running, Role: LEADER
I20260807 22:35:45.849917 3873 consensus_queue.cc:237] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } }
I20260807 22:35:45.850787 3166 catalog_manager.cc:5495] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a reported cstate change: term changed from 0 to 1, leader changed from <none> to 743e793e6dd9451bac7bba45957f185a (127.28.82.195). New cstate: current_term: 1 leader_uuid: "743e793e6dd9451bac7bba45957f185a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "743e793e6dd9451bac7bba45957f185a" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35679 } health_report { overall_health: HEALTHY } } }
I20260807 22:35:45.859061 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.858047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59830 (local address 127.28.82.195:35679)
0807 22:35:45.858108 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:45.858110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:45.858118 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:45.858150 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:45.858151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:45.858162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:45.858242 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:45.858827 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:45.858829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:45.858829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:45.858834 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:45.858847 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:45.858852 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:45.858912 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:45.858979 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 22:35:45.858993 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:35:45.860816 3584 raft_consensus.cc:1261] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 1 FOLLOWER]: Refusing update from remote peer 743e793e6dd9451bac7bba45957f185a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:45.860821 3386 raft_consensus.cc:1261] T 41960e4a58d149ac9ea545ed4b6d74ce P 8f480476145246eca7f22b87f9409295 [term 1 FOLLOWER]: Refusing update from remote peer 743e793e6dd9451bac7bba45957f185a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:45.861178 3873 consensus_queue.cc:1035] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dcd7527d63a4406804308b74fe6e738" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:45.861342 3875 consensus_queue.cc:1035] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f480476145246eca7f22b87f9409295" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:45.864254 3879 mvcc.cc:205] Tried to move back new op lower bound from 7316038229443457024 to 7316038229401915392. Current Snapshot: MvccSnapshot[applied={T|T < 7316038229443457024}]
I20260807 22:35:45.866132 3166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:45.866230 3166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:45.867025 3166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 477f1bea7cf841178cfcb5970a09d5ca: Sending DeleteTablet for 3 replicas of tablet 41960e4a58d149ac9ea545ed4b6d74ce
I20260807 22:35:45.867374 3556 tablet_service.cc:1514] Processing DeleteTablet for tablet 41960e4a58d149ac9ea545ed4b6d74ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:45 UTC) from {username='slave'} at 127.0.0.1:39486
I20260807 22:35:45.867395 3743 tablet_service.cc:1514] Processing DeleteTablet for tablet 41960e4a58d149ac9ea545ed4b6d74ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:45 UTC) from {username='slave'} at 127.0.0.1:59822
I20260807 22:35:45.867600 3891 tablet_replica.cc:330] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: stopping tablet replica
I20260807 22:35:45.867651 3351 tablet_service.cc:1514] Processing DeleteTablet for tablet 41960e4a58d149ac9ea545ed4b6d74ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:45 UTC) from {username='slave'} at 127.0.0.1:45602
I20260807 22:35:45.867703 3891 raft_consensus.cc:2229] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:45.867764 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 3259
I20260807 22:35:45.867839 3890 tablet_replica.cc:330] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: stopping tablet replica
I20260807 22:35:45.867915 3890 raft_consensus.cc:2229] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:45.867992 3890 raft_consensus.cc:2258] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:45.868125 3891 raft_consensus.cc:2258] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:45.872665 3890 ts_tablet_manager.cc:1905] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:45.873487 3891 ts_tablet_manager.cc:1905] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:45.876633 3890 ts_tablet_manager.cc:1918] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:45.876710 3890 log.cc:1196] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/41960e4a58d149ac9ea545ed4b6d74ce
I20260807 22:35:45.876986 3890 ts_tablet_manager.cc:1939] T 41960e4a58d149ac9ea545ed4b6d74ce P 4dcd7527d63a4406804308b74fe6e738: Deleting consensus metadata
I20260807 22:35:45.877669 3891 ts_tablet_manager.cc:1918] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:45.877741 3891 log.cc:1196] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/41960e4a58d149ac9ea545ed4b6d74ce
I20260807 22:35:45.878007 3891 ts_tablet_manager.cc:1939] T 41960e4a58d149ac9ea545ed4b6d74ce P 743e793e6dd9451bac7bba45957f185a: Deleting consensus metadata
I20260807 22:35:45.878428 3149 catalog_manager.cc:4841] TS 743e793e6dd9451bac7bba45957f185a (127.28.82.195:35679): tablet 41960e4a58d149ac9ea545ed4b6d74ce (table test-table [id=d306bfd676e349498dbcf1cab9180e9b]) successfully deleted
I20260807 22:35:45.884088 3153 catalog_manager.cc:4841] TS 4dcd7527d63a4406804308b74fe6e738 (127.28.82.194:46857): tablet 41960e4a58d149ac9ea545ed4b6d74ce (table test-table [id=d306bfd676e349498dbcf1cab9180e9b]) successfully deleted
W20260807 22:35:45.890712 3149 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:43521 (error 108)
I20260807 22:35:45.890782 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 3462
I20260807 22:35:45.892840 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:45.892703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43521 (local address 127.0.0.1:45612)
0807 22:35:45.892757 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:45.892798 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 22:35:45.892957 3149 catalog_manager.cc:4584] TS 8f480476145246eca7f22b87f9409295 (127.28.82.193:43521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41960e4a58d149ac9ea545ed4b6d74ce: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43521: connect: Connection refused (error 111)
I20260807 22:35:45.899006 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 3657
I20260807 22:35:45.908319 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 3123
2026-08-07T22:35:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (785 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 22:35:45.933959 29003 mini_oidc.cc:152] Starting JWKS server
I20260807 22:35:45.935222 29003 webserver.cc:460] Webserver started at https://127.0.0.1:39693/ using document root <none> and password file <none>
I20260807 22:35:45.935339 29003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:35:45.935822 29003 webserver.cc:460] Webserver started at http://127.0.0.1:34711/ using document root <none> and password file <none>
2026-08-07T22:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:45Z Disabled control of system clock
I20260807 22:35:45.945341 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39707
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:44525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39707
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39693/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/testchainca.cert with env {}
W20260807 22:35:46.034680 3904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:46.034883 3904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:46.034904 3904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:46.036067 3904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:46.036113 3904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:46.036125 3904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:46.036137 3904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:46.036156 3904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:35:46.036170 3904 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:35:46.036180 3904 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39693/jwks.json
I20260807 22:35:46.037798 3904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:44525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39693/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:46.038064 3904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:46.038308 3904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:46.043493 3910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:46.043512 3914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:46.043551 3911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:46.043848 3904 server_base.cc:1034] running on GCE node
I20260807 22:35:46.044112 3904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:46.044452 3904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:46.045603 3904 hybrid_clock.cc:648] HybridClock initialized: now 1786142146045586 us; error 42 us; skew 500 ppm
I20260807 22:35:46.046963 3904 webserver.cc:460] Webserver started at http://127.28.82.254:44483/ using document root <none> and password file <none>
I20260807 22:35:46.047223 3904 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:46.047315 3904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:46.047466 3904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:46.048513 3904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "555e7d3b30474b519aba7042eef0283f"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "707e507db5a80b95a415fb95914a81b0"
server_key_iv: "7b68d5f97d8da0c5c02b2d63e6976627"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.048897 3904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "555e7d3b30474b519aba7042eef0283f"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "707e507db5a80b95a415fb95914a81b0"
server_key_iv: "7b68d5f97d8da0c5c02b2d63e6976627"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.050305 3904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:46.051187 3922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:46.051388 3904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:46.051491 3904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "555e7d3b30474b519aba7042eef0283f"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "707e507db5a80b95a415fb95914a81b0"
server_key_iv: "7b68d5f97d8da0c5c02b2d63e6976627"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.051597 3904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:46.060271 3904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:46.060546 3904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:46.060700 3904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:46.065328 3904 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:39707
I20260807 22:35:46.065362 3995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:39707 every 8 connection(s)
I20260807 22:35:46.065708 3904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:46.066751 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:46.068938 3996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f: Bootstrap starting.
I20260807 22:35:46.069481 3996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:46.069734 3996 log.cc:824] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:46.070218 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 3904
I20260807 22:35:46.070364 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:46.070456 3996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f: No bootstrap required, opened a new log
I20260807 22:35:46.070617 29003 external_mini_cluster.cc:1428] Setting key 5a547a579f8221bf8e3fd1bfbb60ab9a
I20260807 22:35:46.073374 3996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555e7d3b30474b519aba7042eef0283f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39707 } }
I20260807 22:35:46.073535 3996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:46.073565 3996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 555e7d3b30474b519aba7042eef0283f, State: Initialized, Role: FOLLOWER
I20260807 22:35:46.073702 3996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555e7d3b30474b519aba7042eef0283f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39707 } }
I20260807 22:35:46.073825 3996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:46.073882 3996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:46.073938 3996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:46.074559 3996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555e7d3b30474b519aba7042eef0283f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39707 } }
I20260807 22:35:46.074688 3996 leader_election.cc:302] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 555e7d3b30474b519aba7042eef0283f; no voters:
I20260807 22:35:46.074859 3996 leader_election.cc:288] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:46.074915 4005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:46.075068 4005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [term 1 LEADER]: Becoming Leader. State: Replica: 555e7d3b30474b519aba7042eef0283f, State: Running, Role: LEADER
I20260807 22:35:46.075156 3996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:46.075157 4005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555e7d3b30474b519aba7042eef0283f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39707 } }
I20260807 22:35:46.075960 4008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "555e7d3b30474b519aba7042eef0283f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555e7d3b30474b519aba7042eef0283f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39707 } } }
I20260807 22:35:46.076052 4008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:46.076318 4006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 555e7d3b30474b519aba7042eef0283f. Latest consensus state: current_term: 1 leader_uuid: "555e7d3b30474b519aba7042eef0283f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "555e7d3b30474b519aba7042eef0283f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39707 } } }
I20260807 22:35:46.076452 4006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:46.077036 4015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:46.077373 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:46.071109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.28.82.254:39707)
0807 22:35:46.071376 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:46.071385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:46.071417 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:35:46.071926 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:46.071934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:46.072010 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:46.072160 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:46.072322 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.073717 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.074727 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.075398 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.075551 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.076382 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:46.076386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:46.076386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:46.076590 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:46.076622 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:46.076633 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:46.076722 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:46.077055 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 22:35:46.077137 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":134,"threads_started":1}
I20260807 22:35:46.077751 4015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:46.078240 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:35:46.082924 4015 catalog_manager.cc:1347] Generated new cluster ID: 2cab46f665fa41318be860716a270bf3
I20260807 22:35:46.082988 4015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:46.101666 4015 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:46.102238 4015 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:46.108793 4015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 555e7d3b30474b519aba7042eef0283f: Generated new TSK 0
I20260807 22:35:46.108996 4015 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 22:35:46.144773 4040 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 22:35:46.144922 4040 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:35:46.145040 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:46.121894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.28.82.254:39707)
0807 22:35:46.121945 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:46.121947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:46.122057 (+ 110us) server_negotiation.cc:407] Connection header received
0807 22:35:46.122162 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:46.122163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:46.122174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:46.122228 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:35:46.122375 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.122932 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.124058 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.124264 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.124322 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.124608 (+ 286us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:35:46.144737 (+ 20129us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 22:35:46.144936 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 22:35:46.145118 4004 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 22:35:46.145457 4038 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:46.121774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39707 (local address 127.0.0.1:42976)
0807 22:35:46.121979 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:46.121990 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:46.122062 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:46.122243 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:46.122244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:46.122246 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:35:46.122315 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:46.122318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.123016 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.123017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:46.123960 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:46.123962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.124361 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.124362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:46.124465 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.124490 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:35:46.144896 (+ 20406us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 22:35:46.145028 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.82.254:39707: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":120,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.28.82.254:39707: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 22:35:46.145962 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 3904
2026-08-07T22:35:46Z chronyd exiting
I20260807 22:35:46.193677 29003 test_util.cc:175] -----------------------------------------------
I20260807 22:35:46.193763 29003 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786142120597695-29003-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (260 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 22:35:46.194609 29003 mini_oidc.cc:152] Starting JWKS server
I20260807 22:35:46.195289 29003 webserver.cc:460] Webserver started at https://127.0.0.1:39485/ using document root <none> and password file <none>
I20260807 22:35:46.195363 29003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:35:46.195521 29003 webserver.cc:460] Webserver started at http://127.0.0.1:35883/ using document root <none> and password file <none>
2026-08-07T22:35:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:46Z Disabled control of system clock
I20260807 22:35:46.205057 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37303
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:35921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:37303
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39485/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/testchainca.cert with env {}
W20260807 22:35:46.288185 4047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:46.288405 4047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:46.288429 4047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:46.289620 4047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:46.289675 4047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:46.289688 4047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:46.289701 4047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:46.289713 4047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:35:46.289726 4047 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:35:46.289737 4047 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39485/jwks.json
I20260807 22:35:46.291600 4047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:35921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:37303
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39485/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:46.291929 4047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:46.292168 4047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:46.297405 4054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:46.297354 4053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:46.297360 4057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:46.297763 4047 server_base.cc:1034] running on GCE node
I20260807 22:35:46.298077 4047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:46.298390 4047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:46.299582 4047 hybrid_clock.cc:648] HybridClock initialized: now 1786142146299562 us; error 63 us; skew 500 ppm
I20260807 22:35:46.300974 4047 webserver.cc:460] Webserver started at http://127.28.82.254:41283/ using document root <none> and password file <none>
I20260807 22:35:46.301249 4047 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:46.301324 4047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:46.301479 4047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:46.302663 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "05890016ca92426593ebf2b67e7a592d"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "c0ba9bd03713859c5ecd792877ec7d0e"
server_key_iv: "10c654d9cc97319e087c082beb05b024"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.303048 4047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "05890016ca92426593ebf2b67e7a592d"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "c0ba9bd03713859c5ecd792877ec7d0e"
server_key_iv: "10c654d9cc97319e087c082beb05b024"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.304476 4047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:35:46.305408 4064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:46.305665 4047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:46.305794 4047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "05890016ca92426593ebf2b67e7a592d"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "c0ba9bd03713859c5ecd792877ec7d0e"
server_key_iv: "10c654d9cc97319e087c082beb05b024"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.305892 4047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:46.324419 4047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:46.324831 4047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:46.325011 4047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:46.329241 4047 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:37303
I20260807 22:35:46.329275 4142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:37303 every 8 connection(s)
I20260807 22:35:46.329655 4047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:46.329952 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 4047
I20260807 22:35:46.330068 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:46.330273 29003 external_mini_cluster.cc:1428] Setting key ea90b1fa1d39afb674e753025dc65724
I20260807 22:35:46.330564 4143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:46.333508 4143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d: Bootstrap starting.
I20260807 22:35:46.334110 4143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:46.334444 4143 log.cc:824] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:46.335242 4143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d: No bootstrap required, opened a new log
I20260807 22:35:46.336488 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:46.331306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33338 (local address 127.28.82.254:37303)
0807 22:35:46.331508 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:46.331516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:46.331546 (+ 30us) server_negotiation.cc:407] Connection header received
0807 22:35:46.332090 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:46.332096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:46.332156 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:46.332311 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:46.332528 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.333402 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.334276 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.334855 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.335008 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.335768 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:46.335773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:46.335773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:46.335978 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:46.336041 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:46.336058 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:46.336134 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:46.336200 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 22:35:46.336277 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260807 22:35:46.337568 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:35:46.337664 4143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05890016ca92426593ebf2b67e7a592d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37303 } }
I20260807 22:35:46.337819 4143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:46.337855 4143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05890016ca92426593ebf2b67e7a592d, State: Initialized, Role: FOLLOWER
I20260807 22:35:46.337963 4143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05890016ca92426593ebf2b67e7a592d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37303 } }
I20260807 22:35:46.338033 4143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:46.338065 4143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:46.338112 4143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:46.338757 4143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05890016ca92426593ebf2b67e7a592d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37303 } }
I20260807 22:35:46.338847 4143 leader_election.cc:302] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05890016ca92426593ebf2b67e7a592d; no voters:
I20260807 22:35:46.338965 4143 leader_election.cc:288] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:46.339044 4149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:46.339193 4149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [term 1 LEADER]: Becoming Leader. State: Replica: 05890016ca92426593ebf2b67e7a592d, State: Running, Role: LEADER
I20260807 22:35:46.339241 4143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:46.339309 4149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05890016ca92426593ebf2b67e7a592d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37303 } }
I20260807 22:35:46.339658 4152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05890016ca92426593ebf2b67e7a592d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05890016ca92426593ebf2b67e7a592d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37303 } } }
I20260807 22:35:46.339659 4153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05890016ca92426593ebf2b67e7a592d. Latest consensus state: current_term: 1 leader_uuid: "05890016ca92426593ebf2b67e7a592d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05890016ca92426593ebf2b67e7a592d" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37303 } } }
I20260807 22:35:46.339881 4153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:46.339763 4152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:46.340085 4158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:46.340605 4158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:46.342382 4158 catalog_manager.cc:1347] Generated new cluster ID: 92c1595360914a83b5cf50278740fd68
I20260807 22:35:46.342440 4158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:46.348480 4158 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:46.349167 4158 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:46.360947 4158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05890016ca92426593ebf2b67e7a592d: Generated new TSK 0
I20260807 22:35:46.361156 4158 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 22:35:46.382259 4180 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 22:35:46.382351 4180 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 22:35:46.382719 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:46.368120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33348 (local address 127.28.82.254:37303)
0807 22:35:46.368166 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:46.368168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:46.368274 (+ 106us) server_negotiation.cc:407] Connection header received
0807 22:35:46.368328 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:46.368329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:46.368340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:46.368404 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:35:46.368549 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.369032 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.370234 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.370455 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.370592 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.370804 (+ 212us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:35:46.382529 (+ 11725us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 22:35:46.382668 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33348: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 22:35:46.382792 4146 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33348: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 22:35:46.382894 4179 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:46.368021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37303 (local address 127.0.0.1:33348)
0807 22:35:46.368185 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:46.368197 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:46.368263 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:46.368417 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:46.368418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:46.368419 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:35:46.368479 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:46.368481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.369131 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.369132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:46.370122 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:46.370125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.370612 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.370613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:46.370681 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.370687 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:35:46.382654 (+ 11967us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 22:35:46.382788 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.82.254:37303: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260807 22:35:46.383181 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 4047
2026-08-07T22:35:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 22:35:46.431134 29003 mini_oidc.cc:152] Starting JWKS server
I20260807 22:35:46.431849 29003 webserver.cc:460] Webserver started at https://127.0.0.1:36191/ using document root <none> and password file <none>
I20260807 22:35:46.431891 29003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 22:35:46.432066 29003 webserver.cc:460] Webserver started at http://127.0.0.1:35593/ using document root <none> and password file <none>
2026-08-07T22:35:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:46Z Disabled control of system clock
I20260807 22:35:46.441972 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39365
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39365
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36191/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 22:35:46.523715 4189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:46.523921 4189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:46.523941 4189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:46.525115 4189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:46.525156 4189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:46.525169 4189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:46.525182 4189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:46.525236 4189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:35:46.525254 4189 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:35:46.525280 4189 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36191/jwks.json
I20260807 22:35:46.526695 4189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39365
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36191/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:46.526947 4189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:46.527240 4189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:46.532228 4195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:46.532194 4196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:46.532405 4189 server_base.cc:1034] running on GCE node
W20260807 22:35:46.532194 4198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:46.532723 4189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:46.533021 4189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:46.534190 4189 hybrid_clock.cc:648] HybridClock initialized: now 1786142146534155 us; error 52 us; skew 500 ppm
I20260807 22:35:46.535543 4189 webserver.cc:460] Webserver started at http://127.28.82.254:45635/ using document root <none> and password file <none>
I20260807 22:35:46.535804 4189 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:46.535884 4189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:46.536024 4189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:46.537070 4189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "d2f1bfcc45474c65b89f1f1fd0454f52"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "54a3d3e8435edee7aaf929a223a8695a"
server_key_iv: "3f9a6dcc3638e15c3c65f2f06a363a52"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.537439 4189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "d2f1bfcc45474c65b89f1f1fd0454f52"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "54a3d3e8435edee7aaf929a223a8695a"
server_key_iv: "3f9a6dcc3638e15c3c65f2f06a363a52"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.538810 4189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:46.540352 4207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:46.540529 4189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:35:46.540611 4189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "d2f1bfcc45474c65b89f1f1fd0454f52"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "54a3d3e8435edee7aaf929a223a8695a"
server_key_iv: "3f9a6dcc3638e15c3c65f2f06a363a52"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.540676 4189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:46.568835 4189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:46.569123 4189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:46.569231 4189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:46.573428 4189 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:39365
I20260807 22:35:46.573464 4283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:39365 every 8 connection(s)
I20260807 22:35:46.573854 4189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:46.574589 4284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:46.576495 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 4189
I20260807 22:35:46.576615 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:46.576815 29003 external_mini_cluster.cc:1428] Setting key 7e89f9c26974f4cd80d3038809824370
I20260807 22:35:46.577904 4284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52: Bootstrap starting.
I20260807 22:35:46.578738 4284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:46.579061 4284 log.cc:824] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:46.579864 4284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52: No bootstrap required, opened a new log
I20260807 22:35:46.582279 4284 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f1bfcc45474c65b89f1f1fd0454f52" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39365 } }
I20260807 22:35:46.582511 4284 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:46.582569 4284 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f1bfcc45474c65b89f1f1fd0454f52, State: Initialized, Role: FOLLOWER
I20260807 22:35:46.582707 4284 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f1bfcc45474c65b89f1f1fd0454f52" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39365 } }
I20260807 22:35:46.582798 4284 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:46.582842 4284 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:46.582890 4284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:46.583356 4288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:46.577391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.28.82.254:39365)
0807 22:35:46.577617 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:46.577625 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:46.577654 (+ 29us) server_negotiation.cc:407] Connection header received
0807 22:35:46.578129 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:46.578136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:46.578202 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:46.578321 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:46.578762 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.580068 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.581128 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.581567 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.581666 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.582451 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:46.582455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:46.582455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:46.582752 (+ 297us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:46.582791 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:46.582825 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:46.582898 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:46.583014 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 22:35:46.583131 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260807 22:35:46.583508 4284 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f1bfcc45474c65b89f1f1fd0454f52" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39365 } }
I20260807 22:35:46.583618 4284 leader_election.cc:302] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2f1bfcc45474c65b89f1f1fd0454f52; no voters:
I20260807 22:35:46.583779 4284 leader_election.cc:288] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:46.583863 4290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:46.584030 4284 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:46.584194 4290 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [term 1 LEADER]: Becoming Leader. State: Replica: d2f1bfcc45474c65b89f1f1fd0454f52, State: Running, Role: LEADER
I20260807 22:35:46.584367 4290 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f1bfcc45474c65b89f1f1fd0454f52" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39365 } }
I20260807 22:35:46.584474 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:35:46.584889 4291 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2f1bfcc45474c65b89f1f1fd0454f52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f1bfcc45474c65b89f1f1fd0454f52" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39365 } } }
I20260807 22:35:46.584982 4291 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:46.585131 4292 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f1bfcc45474c65b89f1f1fd0454f52. Latest consensus state: current_term: 1 leader_uuid: "d2f1bfcc45474c65b89f1f1fd0454f52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f1bfcc45474c65b89f1f1fd0454f52" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39365 } } }
I20260807 22:35:46.585218 4292 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:46.585346 4301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:46.585947 4301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:46.587714 4301 catalog_manager.cc:1347] Generated new cluster ID: 0137fb6663d44af68e3ff33c71976529
I20260807 22:35:46.587790 4301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:46.609407 4301 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:46.609972 4301 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:46.617658 4301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2f1bfcc45474c65b89f1f1fd0454f52: Generated new TSK 0
I20260807 22:35:46.617861 4301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:35:46.635275 4288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:46.631674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.28.82.254:39365)
0807 22:35:46.631724 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:46.631730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:46.631788 (+ 58us) server_negotiation.cc:407] Connection header received
0807 22:35:46.631905 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:46.631905 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:46.631924 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:46.631986 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:35:46.632128 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.632640 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.633727 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.633925 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.633989 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.634173 (+ 184us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:35:46.635020 (+ 847us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36191/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 22:35:46.635191 (+ 171us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36191/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 22:35:46.635332 4320 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:46.631532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39365 (local address 127.0.0.1:39602)
0807 22:35:46.631729 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:46.631741 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:46.631807 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:46.632003 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:46.632004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:46.632007 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:35:46.632067 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:46.632069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.632738 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.632739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:46.633638 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:46.633641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.634007 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.634008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:46.634067 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.634072 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:35:46.635142 (+ 1070us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36191/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 22:35:46.635238 (+ 96us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.82.254:39365: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36191/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":130,"threads_started":1}
W20260807 22:35:46.635349 4288 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36191/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260807 22:35:46.635665 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 4189
E20260807 22:35:46.653049 4321 webserver.cc:533] Webserver: sslize failed:
E20260807 22:35:46.653131 4321 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T22:35:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T22:35:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:46Z Disabled control of system clock
I20260807 22:35:46.689482 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:35755
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:36387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:35755
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/testchainca.cert with env {}
W20260807 22:35:46.772400 4328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:46.772604 4328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:46.772627 4328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:46.773734 4328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:46.773770 4328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:46.773783 4328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:46.773795 4328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:46.773808 4328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 22:35:46.773820 4328 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 22:35:46.773831 4328 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 22:35:46.775274 4328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:35755
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:35755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:46.775475 4328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:46.775638 4328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:46.780512 4337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:46.780521 4336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:46.780812 4328 server_base.cc:1034] running on GCE node
W20260807 22:35:46.780582 4340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:46.781168 4328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:46.781488 4328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:46.782656 4328 hybrid_clock.cc:648] HybridClock initialized: now 1786142146782639 us; error 51 us; skew 500 ppm
I20260807 22:35:46.783927 4328 webserver.cc:460] Webserver started at http://127.28.82.254:44927/ using document root <none> and password file <none>
I20260807 22:35:46.784192 4328 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:46.784265 4328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:46.784415 4328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:46.785437 4328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "d08e683329ce4709bcbeaf3051ece90f"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "c84e73b0a9c8dbe77a281cbad7e07bf7"
server_key_iv: "b17b815545b4cb67bb5b9ffcb9eb23e3"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.785822 4328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "d08e683329ce4709bcbeaf3051ece90f"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "c84e73b0a9c8dbe77a281cbad7e07bf7"
server_key_iv: "b17b815545b4cb67bb5b9ffcb9eb23e3"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.787195 4328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:46.787936 4348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:46.788113 4328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:46.788214 4328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "d08e683329ce4709bcbeaf3051ece90f"
format_stamp: "Formatted at 2026-08-07 22:35:46 on dist-test-slave-7kzw"
server_key: "c84e73b0a9c8dbe77a281cbad7e07bf7"
server_key_iv: "b17b815545b4cb67bb5b9ffcb9eb23e3"
server_key_version: "encryptionkey@0"
I20260807 22:35:46.788300 4328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:46.807453 4328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:46.807818 4328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:46.807992 4328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:46.812503 4328 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:35755
I20260807 22:35:46.812563 4427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:35755 every 8 connection(s)
I20260807 22:35:46.812971 4328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:46.813979 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 4328
I20260807 22:35:46.814015 4428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:46.814100 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:46.814287 29003 external_mini_cluster.cc:1428] Setting key e264599a83e2f1cd50023690fdca51dd
I20260807 22:35:46.817303 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f: Bootstrap starting.
I20260807 22:35:46.818073 4428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:46.818380 4428 log.cc:824] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:46.819288 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f: No bootstrap required, opened a new log
I20260807 22:35:46.820164 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:46.814872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50968 (local address 127.28.82.254:35755)
0807 22:35:46.815091 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:46.815098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:46.815123 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:35:46.815635 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:46.815642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:46.815703 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:46.815818 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:46.816046 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.816888 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.817967 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.818409 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.818586 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.819306 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:46.819310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:46.819310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:35:46.819522 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:46.819555 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:35:46.819567 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:46.819648 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:46.819833 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 22:35:46.819955 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260807 22:35:46.821331 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:35:46.821599 4428 raft_consensus.cc:348] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08e683329ce4709bcbeaf3051ece90f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35755 } }
I20260807 22:35:46.821761 4428 raft_consensus.cc:374] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:46.821801 4428 raft_consensus.cc:729] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08e683329ce4709bcbeaf3051ece90f, State: Initialized, Role: FOLLOWER
I20260807 22:35:46.821928 4428 consensus_queue.cc:260] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08e683329ce4709bcbeaf3051ece90f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35755 } }
I20260807 22:35:46.822001 4428 raft_consensus.cc:388] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:46.822038 4428 raft_consensus.cc:482] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:46.822089 4428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:46.822813 4428 raft_consensus.cc:504] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08e683329ce4709bcbeaf3051ece90f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35755 } }
I20260807 22:35:46.822938 4428 leader_election.cc:302] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d08e683329ce4709bcbeaf3051ece90f; no voters:
I20260807 22:35:46.823122 4428 leader_election.cc:288] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:46.823164 4435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:46.823344 4435 raft_consensus.cc:686] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [term 1 LEADER]: Becoming Leader. State: Replica: d08e683329ce4709bcbeaf3051ece90f, State: Running, Role: LEADER
I20260807 22:35:46.823385 4428 sys_catalog.cc:564] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:46.823438 4435 consensus_queue.cc:237] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08e683329ce4709bcbeaf3051ece90f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35755 } }
I20260807 22:35:46.823849 4437 sys_catalog.cc:455] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d08e683329ce4709bcbeaf3051ece90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08e683329ce4709bcbeaf3051ece90f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35755 } } }
I20260807 22:35:46.823844 4438 sys_catalog.cc:455] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d08e683329ce4709bcbeaf3051ece90f. Latest consensus state: current_term: 1 leader_uuid: "d08e683329ce4709bcbeaf3051ece90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08e683329ce4709bcbeaf3051ece90f" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35755 } } }
I20260807 22:35:46.823951 4437 sys_catalog.cc:458] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:46.823959 4438 sys_catalog.cc:458] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:46.824241 4449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:46.824702 4449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:46.827453 4449 catalog_manager.cc:1347] Generated new cluster ID: ca0b3d1ff08f4d459ea153ffceb61050
I20260807 22:35:46.827512 4449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:46.843076 4449 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:46.843793 4449 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:46.848627 4449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d08e683329ce4709bcbeaf3051ece90f: Generated new TSK 0
I20260807 22:35:46.848800 4449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:35:47.018407 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:46.854972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.28.82.254:35755)
0807 22:35:46.855037 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:46.855039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:46.855146 (+ 107us) server_negotiation.cc:407] Connection header received
0807 22:35:46.855196 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:46.855197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:46.855211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:46.855292 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0807 22:35:46.855478 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.856287 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.857111 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.857446 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.857519 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.857670 (+ 151us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 22:35:47.017933 (+160263us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 22:35:47.018338 (+ 405us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50974: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 22:35:47.018519 4476 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:46.854869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35755 (local address 127.0.0.1:50974)
0807 22:35:46.855078 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:46.855087 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:46.855141 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:46.855324 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:46.855325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:46.855327 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 22:35:46.855395 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:46.855399 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.856401 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.856402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:46.857024 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:46.857026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:46.857530 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:46.857531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:46.857575 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:46.857580 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 22:35:47.018246 (+160666us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 22:35:47.018377 (+ 131us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.82.254:35755: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":135,"threads_started":1}
W20260807 22:35:47.018596 4433 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50974: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260807 22:35:47.018958 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 4328
2026-08-07T22:35:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (364 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:47 dist-test-slave-7kzw krb5kdc[4492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:47 dist-test-slave-7kzw krb5kdc[4492](info): set up 2 sockets
Aug 07 22:35:47 dist-test-slave-7kzw krb5kdc[4492](info): commencing operation
krb5kdc: starting...
W20260807 22:35:48.445330 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:48 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:35:48.573412 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45127
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:45127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:48.657508 4509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:48.657742 4509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:48.657768 4509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:48.659042 4509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:48.659092 4509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:48.659109 4509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:48.659123 4509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:48.659137 4509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:48.660775 4509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:45127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:48.661056 4509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:48.661291 4509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:48.666275 4520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:48.666234 4518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:48.666246 4517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:48.666538 4509 server_base.cc:1034] running on GCE node
I20260807 22:35:48.666850 4509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:48.667128 4509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:48.668293 4509 hybrid_clock.cc:648] HybridClock initialized: now 1786142148668273 us; error 35 us; skew 500 ppm
Aug 07 22:35:48 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142148, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:48.670825 4509 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:35:48.671240 4509 webserver.cc:460] Webserver started at http://127.28.82.254:36415/ using document root <none> and password file <none>
I20260807 22:35:48.671473 4509 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:48.671545 4509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:48.671698 4509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:48.672744 4509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd"
format_stamp: "Formatted at 2026-08-07 22:35:48 on dist-test-slave-7kzw"
server_key: "a2822d5ee1cea4a5bf6b4974f10ee94f"
server_key_iv: "bb1aadb43f72dfb573779531f72d7fc1"
server_key_version: "encryptionkey@0"
I20260807 22:35:48.673100 4509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd"
format_stamp: "Formatted at 2026-08-07 22:35:48 on dist-test-slave-7kzw"
server_key: "a2822d5ee1cea4a5bf6b4974f10ee94f"
server_key_iv: "bb1aadb43f72dfb573779531f72d7fc1"
server_key_version: "encryptionkey@0"
I20260807 22:35:48.674422 4509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 22:35:48.675196 4530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:48.675359 4509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:48.675462 4509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd"
format_stamp: "Formatted at 2026-08-07 22:35:48 on dist-test-slave-7kzw"
server_key: "a2822d5ee1cea4a5bf6b4974f10ee94f"
server_key_iv: "bb1aadb43f72dfb573779531f72d7fc1"
server_key_version: "encryptionkey@0"
I20260807 22:35:48.675544 4509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:48.687754 4509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:48.689584 4509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:48.689749 4509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:48.693879 4509 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:45127
I20260807 22:35:48.693915 4601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:45127 every 8 connection(s)
I20260807 22:35:48.694273 4509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:48.695215 4603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:48.697424 4603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd: Bootstrap starting.
I20260807 22:35:48.697953 4603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:48.698221 4603 log.cc:824] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:48.698570 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 4509
I20260807 22:35:48.698686 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:48.698892 29003 external_mini_cluster.cc:1428] Setting key 88a80774cbe48e8f9541635edb24c365
I20260807 22:35:48.699015 4603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd: No bootstrap required, opened a new log
I20260807 22:35:48.701299 4603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45127 } }
I20260807 22:35:48.701484 4603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:48.701531 4603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ff00f43b8a4de99f0fe43ed7fdd5dd, State: Initialized, Role: FOLLOWER
I20260807 22:35:48.701684 4603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45127 } }
I20260807 22:35:48.701776 4603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:48.701807 4603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:48.701853 4603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:48.702555 4603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45127 } }
I20260807 22:35:48.702665 4603 leader_election.cc:302] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86ff00f43b8a4de99f0fe43ed7fdd5dd; no voters:
I20260807 22:35:48.702818 4603 leader_election.cc:288] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:48.702895 4610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:48.703033 4603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:48.703054 4610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [term 1 LEADER]: Becoming Leader. State: Replica: 86ff00f43b8a4de99f0fe43ed7fdd5dd, State: Running, Role: LEADER
I20260807 22:35:48.703147 4610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45127 } }
I20260807 22:35:48.703469 4610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86ff00f43b8a4de99f0fe43ed7fdd5dd. Latest consensus state: current_term: 1 leader_uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45127 } } }
I20260807 22:35:48.703563 4610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:48.703480 4614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ff00f43b8a4de99f0fe43ed7fdd5dd" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45127 } } }
I20260807 22:35:48.703711 4614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:48.703831 4621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:48.704360 4621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:48.706570 4621 catalog_manager.cc:1347] Generated new cluster ID: 16c7dc5a01ef42609be905a44f121887
I20260807 22:35:48.706629 4621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:35:48 dist-test-slave-7kzw krb5kdc[4492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:48.711253 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:48.699427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.28.82.254:45127)
0807 22:35:48.699639 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:48.699648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:48.699687 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:35:48.700456 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:48.700465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:48.700527 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:48.700681 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:48.701284 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:48.703012 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:48.704179 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:48.704838 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:48.704942 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:48.707807 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:48.707811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:48.707811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:48.707824 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:48.709546 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:48.709937 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:48.709940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:48.709940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:48.710040 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:48.710263 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:48.710264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:48.710265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:48.710575 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:48.710741 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:48.710860 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 22:35:48.711101 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
I20260807 22:35:48.726725 4621 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:48.727277 4621 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:48.730203 4621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86ff00f43b8a4de99f0fe43ed7fdd5dd: Generated new TSK 0
I20260807 22:35:48.730387 4621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:35:48.761670 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45127
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:48.854547 4640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:48.854787 4640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:48.854828 4640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:48.855980 4640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:48.856050 4640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:48.856096 4640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:48.858125 4640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:48.858392 4640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:48.858692 4640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:48.863978 4650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:48.863991 4651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:48.863976 4654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:48.864701 4640 server_base.cc:1034] running on GCE node
I20260807 22:35:48.864953 4640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:48.865191 4640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:48.866362 4640 hybrid_clock.cc:648] HybridClock initialized: now 1786142148866340 us; error 35 us; skew 500 ppm
Aug 07 22:35:48 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142148, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:48.869007 4640 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:35:48.869452 4640 webserver.cc:460] Webserver started at http://127.28.82.193:35161/ using document root <none> and password file <none>
I20260807 22:35:48.869648 4640 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:48.869689 4640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:48.869784 4640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:48.870858 4640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "8711c44ce64041f7aa6d61ca2ff2e6c2"
format_stamp: "Formatted at 2026-08-07 22:35:48 on dist-test-slave-7kzw"
server_key: "a9d68ae28e12d811b744aba06d0e6344"
server_key_iv: "891bd039875bcaefbb7615d65646f549"
server_key_version: "encryptionkey@0"
I20260807 22:35:48.871167 4640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "8711c44ce64041f7aa6d61ca2ff2e6c2"
format_stamp: "Formatted at 2026-08-07 22:35:48 on dist-test-slave-7kzw"
server_key: "a9d68ae28e12d811b744aba06d0e6344"
server_key_iv: "891bd039875bcaefbb7615d65646f549"
server_key_version: "encryptionkey@0"
I20260807 22:35:48.872473 4640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:48.873147 4666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:48.873329 4640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:48.873421 4640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "8711c44ce64041f7aa6d61ca2ff2e6c2"
format_stamp: "Formatted at 2026-08-07 22:35:48 on dist-test-slave-7kzw"
server_key: "a9d68ae28e12d811b744aba06d0e6344"
server_key_iv: "891bd039875bcaefbb7615d65646f549"
server_key_version: "encryptionkey@0"
I20260807 22:35:48.873476 4640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:48.889356 4640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:48.891453 4640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:48.891567 4640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:48.891758 4640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:48.892092 4640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:48.892128 4640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:48.892151 4640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:48.892199 4640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:48.897913 4640 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:43959
I20260807 22:35:48.898108 4822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:43959 every 8 connection(s)
I20260807 22:35:48.898331 4640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:35:48 dist-test-slave-7kzw krb5kdc[4492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142148, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:48.907454 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 4640
I20260807 22:35:48.907575 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:48.907799 29003 external_mini_cluster.cc:1428] Setting key 83fca0c8a438f23b9d6e818a4724496e
I20260807 22:35:48.908753 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:48.899140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:53421 (local address 127.28.82.254:45127)
0807 22:35:48.899234 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:48.899236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:48.899375 (+ 139us) server_negotiation.cc:407] Connection header received
0807 22:35:48.900468 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:48.900469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:48.900486 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:48.900584 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:48.901357 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:48.901949 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:48.903078 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:48.903294 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:48.903361 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:48.905516 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:48.905517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:48.905517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:48.905524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:48.907037 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:48.907573 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:48.907575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:48.907575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:48.907661 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:48.907990 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:48.907992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:48.907992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:48.908144 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:48.908226 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:48.908551 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 22:35:48.908648 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 22:35:48.908839 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:48.899064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45127 (local address 127.28.82.193:53421)
0807 22:35:48.899240 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:48.899283 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:48.900204 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:48.900768 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:48.900771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:48.901065 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:48.901255 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:48.901259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:48.902045 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:48.902046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:48.902978 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:48.902981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:48.903376 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:48.903377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:48.903474 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:48.903947 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:48.903955 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:48.905428 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:48.907106 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:48.907112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:48.907114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:48.907438 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:48.907762 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:48.907763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:48.907764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:48.907910 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:48.908218 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:48.908226 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:48.908368 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 22:35:48.908492 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 22:35:48.908658 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260807 22:35:48.909636 4823 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45127
I20260807 22:35:48.909747 4823 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:48.909917 4823 heartbeater.cc:511] Master 127.28.82.254:45127 requested a full tablet report, sending...
I20260807 22:35:48.910523 4555 ts_manager.cc:194] Registered new tserver with Master: 8711c44ce64041f7aa6d61ca2ff2e6c2 (127.28.82.193:43959)
I20260807 22:35:48.911154 4555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:53421
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:35:48.934092 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45127
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:49.016142 4833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:49.016366 4833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:49.016388 4833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:49.017549 4833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:49.017592 4833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:49.017618 4833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:49.019102 4833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:49.019357 4833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:49.019598 4833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:49.024766 4844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:49.024765 4841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:49.024842 4833 server_base.cc:1034] running on GCE node
W20260807 22:35:49.024766 4842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:49.025175 4833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:49.025415 4833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:49.026556 4833 hybrid_clock.cc:648] HybridClock initialized: now 1786142149026526 us; error 50 us; skew 500 ppm
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.029362 4833 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:35:49.029847 4833 webserver.cc:460] Webserver started at http://127.28.82.194:33493/ using document root <none> and password file <none>
I20260807 22:35:49.030097 4833 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:49.030172 4833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:49.030345 4833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:49.031474 4833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "ff0e8354b0614e13bf95397a074620ee"
format_stamp: "Formatted at 2026-08-07 22:35:49 on dist-test-slave-7kzw"
server_key: "009add7f3e12aac194e08643f2104190"
server_key_iv: "85256da2c7ad745e590daa8375d9e7f6"
server_key_version: "encryptionkey@0"
I20260807 22:35:49.031878 4833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "ff0e8354b0614e13bf95397a074620ee"
format_stamp: "Formatted at 2026-08-07 22:35:49 on dist-test-slave-7kzw"
server_key: "009add7f3e12aac194e08643f2104190"
server_key_iv: "85256da2c7ad745e590daa8375d9e7f6"
server_key_version: "encryptionkey@0"
I20260807 22:35:49.033334 4833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:49.034086 4855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:49.034288 4833 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:49.034410 4833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "ff0e8354b0614e13bf95397a074620ee"
format_stamp: "Formatted at 2026-08-07 22:35:49 on dist-test-slave-7kzw"
server_key: "009add7f3e12aac194e08643f2104190"
server_key_iv: "85256da2c7ad745e590daa8375d9e7f6"
server_key_version: "encryptionkey@0"
I20260807 22:35:49.034515 4833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:49.044687 4833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:49.046828 4833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:49.046949 4833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:49.047132 4833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:49.047411 4833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:49.047446 4833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:49.047467 4833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:49.047479 4833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:49.053290 4833 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:33369
I20260807 22:35:49.053367 5034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:33369 every 8 connection(s)
I20260807 22:35:49.053714 4833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:49.058959 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 4833
I20260807 22:35:49.059046 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:49.059250 29003 external_mini_cluster.cc:1428] Setting key 2ab0f755143880ebbecaac69d83a6bba
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:49.064000 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:49.054666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:58229 (local address 127.28.82.254:45127)
0807 22:35:49.054744 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:49.054746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:49.054887 (+ 141us) server_negotiation.cc:407] Connection header received
0807 22:35:49.055875 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:49.055876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:49.055887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:49.055943 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:49.056541 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.057033 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.058002 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.058179 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.058231 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:49.060790 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:49.060791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:49.060792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:49.060800 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:49.062577 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.062991 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.062992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.062992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.063064 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.063301 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.063302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.063302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.063457 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:49.063539 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:49.063771 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 22:35:49.063863 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:35:49.064018 5037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:49.054448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45127 (local address 127.28.82.194:58229)
0807 22:35:49.054742 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:49.054801 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:49.055768 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:49.055996 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:49.055999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:49.056271 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:49.056461 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:49.056465 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.057131 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.057132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:49.057919 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:49.057921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.058251 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.058252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:49.058349 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:49.058958 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:49.058966 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:49.060687 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:49.062659 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:49.062665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:49.062666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:49.062910 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:49.063134 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:49.063134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:49.063135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:49.063234 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:49.063525 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:49.063529 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:49.063638 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 22:35:49.063721 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:35:49.063854 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":109,"threads_started":1}
I20260807 22:35:49.064827 5035 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45127
I20260807 22:35:49.064935 5035 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:49.065137 5035 heartbeater.cc:511] Master 127.28.82.254:45127 requested a full tablet report, sending...
I20260807 22:35:49.065590 4555 ts_manager.cc:194] Registered new tserver with Master: ff0e8354b0614e13bf95397a074620ee (127.28.82.194:33369)
I20260807 22:35:49.065982 4555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:58229
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:35:49.089478 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45127
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:49.174010 5040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:49.174284 5040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:49.174325 5040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:49.175684 5040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:49.175756 5040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:49.175802 5040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:49.177397 5040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:49.177678 5040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:49.177923 5040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:49.183159 5050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:49.183162 5049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:49.183333 5040 server_base.cc:1034] running on GCE node
W20260807 22:35:49.183187 5053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:49.183677 5040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:49.183961 5040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:49.185111 5040 hybrid_clock.cc:648] HybridClock initialized: now 1786142149185087 us; error 33 us; skew 500 ppm
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.187834 5040 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:35:49.188300 5040 webserver.cc:460] Webserver started at http://127.28.82.195:37581/ using document root <none> and password file <none>
I20260807 22:35:49.188544 5040 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:49.188618 5040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:49.188787 5040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:49.189906 5040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "a859f26dc87949b4a5f1629ddc620d8a"
format_stamp: "Formatted at 2026-08-07 22:35:49 on dist-test-slave-7kzw"
server_key: "35e3a1c9958b23952c7c703834fc0619"
server_key_iv: "66193b29f73a5629e4eba664930cfa36"
server_key_version: "encryptionkey@0"
I20260807 22:35:49.190310 5040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "a859f26dc87949b4a5f1629ddc620d8a"
format_stamp: "Formatted at 2026-08-07 22:35:49 on dist-test-slave-7kzw"
server_key: "35e3a1c9958b23952c7c703834fc0619"
server_key_iv: "66193b29f73a5629e4eba664930cfa36"
server_key_version: "encryptionkey@0"
I20260807 22:35:49.191838 5040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:49.192687 5066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:49.192902 5040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:49.193018 5040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "a859f26dc87949b4a5f1629ddc620d8a"
format_stamp: "Formatted at 2026-08-07 22:35:49 on dist-test-slave-7kzw"
server_key: "35e3a1c9958b23952c7c703834fc0619"
server_key_iv: "66193b29f73a5629e4eba664930cfa36"
server_key_version: "encryptionkey@0"
I20260807 22:35:49.193111 5040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:49.208832 5040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:49.210997 5040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:49.211190 5040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:49.211441 5040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:49.211776 5040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:49.211838 5040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:49.211889 5040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:49.211922 5040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:49.218232 5040 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:35031
I20260807 22:35:49.218272 5230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:35031 every 8 connection(s)
I20260807 22:35:49.218713 5040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:49.225299 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 5040
I20260807 22:35:49.225382 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:35:49.225603 29003 external_mini_cluster.cc:1428] Setting key 1fc98be3bfa109bf06565a121ed62c33
I20260807 22:35:49.228922 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:49.219544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:35001 (local address 127.28.82.254:45127)
0807 22:35:49.219611 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:49.219614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:49.219857 (+ 243us) server_negotiation.cc:407] Connection header received
0807 22:35:49.220858 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:49.220860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:49.220872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:49.220931 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:49.221697 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.222196 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.223353 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.223547 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.223605 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:49.225932 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:49.225933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:49.225934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:49.225942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:49.227515 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.227929 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.227931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.227931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.227998 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.228266 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.228267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.228268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.228391 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:49.228468 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:49.228740 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 22:35:49.228823 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:35:49.229120 5235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:49.219526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45127 (local address 127.28.82.195:35001)
0807 22:35:49.219712 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:49.219757 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:49.220760 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:49.221015 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:49.221019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:49.221388 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:49.221590 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:49.221595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.222298 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.222299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:49.223265 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:49.223268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.223630 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.223631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:49.223763 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:49.224228 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:49.224237 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:49.225836 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:49.227604 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:49.227610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:49.227612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:49.227851 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:49.228066 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:49.228067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:49.228067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:49.228189 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:49.228459 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:49.228465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:49.228601 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 22:35:49.228704 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 22:35:49.228894 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260807 22:35:49.229641 5232 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45127
I20260807 22:35:49.229722 5232 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:49.229861 5232 heartbeater.cc:511] Master 127.28.82.254:45127 requested a full tablet report, sending...
I20260807 22:35:49.230242 4555 ts_manager.cc:194] Registered new tserver with Master: a859f26dc87949b4a5f1629ddc620d8a (127.28.82.195:35031)
I20260807 22:35:49.230720 4555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:35001
I20260807 22:35:49.237427 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/test-admin.keytab.
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.254994 29003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 22:35:49.256801 5241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:49.255967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.28.82.193:43959)
0807 22:35:49.256171 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:49.256176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:49.256186 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:49.256213 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:49.256215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:49.256229 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:49.256299 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:49.256699 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41420: BlockingRecv error: recv got EOF from 127.0.0.1:41420 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
W20260807 22:35:49.256834 29003 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.257834 29003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.258761 29003 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:49.267325 5241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:49.259283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.28.82.193:43959)
0807 22:35:49.259340 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:49.259342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:49.259368 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:35:49.259433 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:49.259434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:49.259444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:49.259493 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:49.259855 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.260573 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.261303 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.261604 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.261695 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:49.264130 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:49.264141 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:49.264141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:49.264148 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:49.265967 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.266358 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.266361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.266361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.266421 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.266731 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.266732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.266732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.266875 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:49.266986 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:49.267059 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 22:35:49.267137 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 22:35:49.269204 5241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:49.268695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.28.82.193:43959)
0807 22:35:49.268758 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:49.268760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:49.268768 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:49.268815 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:49.268816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:49.268824 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:49.268878 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:49.269142 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41442: BlockingRecv error: recv got EOF from 127.0.0.1:41442 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 22:35:49.269351 29003 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5cc' of type FILE
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.270350 29003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.271322 29003 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:49.278788 5241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:49.271878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.28.82.193:43959)
0807 22:35:49.271934 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:49.271936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:49.271976 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:35:49.272050 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:49.272051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:49.272059 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:49.272129 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:49.272489 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.273001 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.273714 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.273912 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.273988 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:49.276096 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:49.276098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:49.276098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:49.276105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:49.277610 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.277964 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.277965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.277965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.278031 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.278210 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.278210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.278210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.278301 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:49.278376 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:49.278615 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 22:35:49.278686 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 22:35:49.280215 5241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:49.279751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.28.82.193:43959)
0807 22:35:49.279785 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:49.279787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:49.279826 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:35:49.279873 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:49.279874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:49.279880 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:49.279931 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:49.280152 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41448: BlockingRecv error: recv got EOF from 127.0.0.1:41448 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 22:35:49.280390 29003 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786142120597695-29003-0/krb5kdc/krb5cc)
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.281313 29003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:49.282150 29003 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[4492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:35:49.288966 5241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:49.282631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.28.82.193:43959)
0807 22:35:49.282686 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:49.282688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:49.282763 (+ 75us) server_negotiation.cc:407] Connection header received
0807 22:35:49.282805 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:49.282806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:49.282813 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:49.282858 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:49.283185 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.283615 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.284274 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:49.284480 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:49.284545 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:49.286580 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:49.286582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:49.286582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:49.286588 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:49.287989 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.288294 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.288295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.288295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.288345 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:49.288509 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:49.288510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:49.288510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:49.288593 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:49.288662 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:49.288774 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 22:35:49.288827 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:35:49.289620 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 4640
I20260807 22:35:49.302273 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 4833
I20260807 22:35:49.311937 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 5040
I20260807 22:35:49.319538 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 4509
2026-08-07T22:35:49Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2109 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[5255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[5255](info): set up 2 sockets
Aug 07 22:35:49 dist-test-slave-7kzw krb5kdc[5255](info): commencing operation
krb5kdc: starting...
W20260807 22:35:50.543262 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:50 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.254.keytab.
Entry for principal oryx/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.254.keytab.
I20260807 22:35:50.669093 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:36739
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.254.keytab
--principal=oryx/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:36739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:50.752132 5274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:50.752346 5274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:50.752368 5274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:50.753585 5274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:50.753628 5274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:50.753641 5274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:50.753654 5274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:50.753705 5274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:50.755177 5274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:36739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.254.keytab
--principal=oryx/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:36739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:50.755434 5274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:50.755671 5274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:50.760679 5289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:50.760800 5274 server_base.cc:1034] running on GCE node
W20260807 22:35:50.760695 5283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:50.760703 5287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:50.761186 5274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:50.761435 5274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:50.762611 5274 hybrid_clock.cc:648] HybridClock initialized: now 1786142150762589 us; error 39 us; skew 500 ppm
Aug 07 22:35:50 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:50.765175 5274 init.cc:377] Logged in from keytab as oryx/127.28.82.254@KRBTEST.COM (short username oryx)
I20260807 22:35:50.765547 5274 webserver.cc:460] Webserver started at http://127.28.82.254:35819/ using document root <none> and password file <none>
I20260807 22:35:50.765728 5274 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:50.765769 5274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:50.765867 5274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:50.766920 5274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "21af200769d44c058aa2fb4efd2fd9a2"
format_stamp: "Formatted at 2026-08-07 22:35:50 on dist-test-slave-7kzw"
server_key: "ca7bbd45113cb5dce50f7ff6dff30e31"
server_key_iv: "2c250d064d1876e7f3961aa2c3a995da"
server_key_version: "encryptionkey@0"
I20260807 22:35:50.767230 5274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "21af200769d44c058aa2fb4efd2fd9a2"
format_stamp: "Formatted at 2026-08-07 22:35:50 on dist-test-slave-7kzw"
server_key: "ca7bbd45113cb5dce50f7ff6dff30e31"
server_key_iv: "2c250d064d1876e7f3961aa2c3a995da"
server_key_version: "encryptionkey@0"
I20260807 22:35:50.768529 5274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:50.769255 5300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:50.769466 5274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:50.769543 5274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "21af200769d44c058aa2fb4efd2fd9a2"
format_stamp: "Formatted at 2026-08-07 22:35:50 on dist-test-slave-7kzw"
server_key: "ca7bbd45113cb5dce50f7ff6dff30e31"
server_key_iv: "2c250d064d1876e7f3961aa2c3a995da"
server_key_version: "encryptionkey@0"
I20260807 22:35:50.769634 5274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:50.793411 5274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:50.795819 5274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:50.795966 5274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:50.800060 5274 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:36739
I20260807 22:35:50.800284 5381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:36739 every 8 connection(s)
I20260807 22:35:50.800519 5274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:50.801380 5383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:50.803797 5383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: Bootstrap starting.
I20260807 22:35:50.804385 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 5274
I20260807 22:35:50.804472 5383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:50.804484 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:50.804694 29003 external_mini_cluster.cc:1428] Setting key e051976f3b169ff6cf2555dcf5d9241b
I20260807 22:35:50.804814 5383 log.cc:824] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:50.805636 5383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: No bootstrap required, opened a new log
I20260807 22:35:50.807947 5383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } }
I20260807 22:35:50.808146 5383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:50.808185 5383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21af200769d44c058aa2fb4efd2fd9a2, State: Initialized, Role: FOLLOWER
I20260807 22:35:50.808306 5383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } }
I20260807 22:35:50.808406 5383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:50.808439 5383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:50.808478 5383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:50.809125 5383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } }
I20260807 22:35:50.809214 5383 leader_election.cc:302] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21af200769d44c058aa2fb4efd2fd9a2; no voters:
I20260807 22:35:50.809428 5383 leader_election.cc:288] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:50.809478 5392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:50.809661 5392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 1 LEADER]: Becoming Leader. State: Replica: 21af200769d44c058aa2fb4efd2fd9a2, State: Running, Role: LEADER
I20260807 22:35:50.809731 5383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:50.809762 5392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } }
I20260807 22:35:50.810094 5395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } }
I20260807 22:35:50.810102 5394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21af200769d44c058aa2fb4efd2fd9a2. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } }
I20260807 22:35:50.810334 5394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:50.810299 5395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:50.810597 5405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:50.811205 5405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 22:35:50 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:50.813477 5405 catalog_manager.cc:1347] Generated new cluster ID: 951f417b1e7f44d5b8886c82cc9e9e6e
I20260807 22:35:50.813534 5405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:50.816705 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:50.805228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41230 (local address 127.28.82.254:36739)
0807 22:35:50.805390 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:50.805398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:50.805437 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:35:50.805928 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:50.805935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:50.805988 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:50.806105 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:50.806963 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:50.807901 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:50.808947 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:50.809482 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:50.809599 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:50.813350 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:50.813354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:50.813354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:50.813367 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:50.815161 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:50.815550 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:50.815552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:50.815553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:50.815645 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:50.815864 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:50.815865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:50.815866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:50.816108 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:50.816301 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:50.816366 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 22:35:50.816503 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260807 22:35:50.851816 5405 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:50.852403 5405 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:50.857239 5405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: Generated new TSK 0
I20260807 22:35:50.857430 5405 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab.
Entry for principal oryx/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab.
I20260807 22:35:50.887481 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36739
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab
--principal=oryx/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:50.970724 5420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:50.970930 5420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:50.970953 5420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:50.972203 5420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:50.972245 5420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:50.972301 5420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:50.973887 5420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab
--principal=oryx/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:50.974126 5420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:50.974380 5420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:50.979408 5431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:50.979471 5428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:50.979497 5429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:50.979696 5420 server_base.cc:1034] running on GCE node
I20260807 22:35:50.979903 5420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:50.980140 5420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:50.981304 5420 hybrid_clock.cc:648] HybridClock initialized: now 1786142150981290 us; error 28 us; skew 500 ppm
Aug 07 22:35:50 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:50.983928 5420 init.cc:377] Logged in from keytab as oryx/127.28.82.193@KRBTEST.COM (short username oryx)
I20260807 22:35:50.984383 5420 webserver.cc:460] Webserver started at http://127.28.82.193:41303/ using document root <none> and password file <none>
I20260807 22:35:50.984611 5420 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:50.984681 5420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:50.984839 5420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:50.985880 5420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "8b06e445ec1c4b1496ae351e79d8f793"
format_stamp: "Formatted at 2026-08-07 22:35:50 on dist-test-slave-7kzw"
server_key: "498ca1300c5d8fa00f4d7bd0041ece5f"
server_key_iv: "98c55c5f1d51669d242387d5fcd19a11"
server_key_version: "encryptionkey@0"
I20260807 22:35:50.986263 5420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "8b06e445ec1c4b1496ae351e79d8f793"
format_stamp: "Formatted at 2026-08-07 22:35:50 on dist-test-slave-7kzw"
server_key: "498ca1300c5d8fa00f4d7bd0041ece5f"
server_key_iv: "98c55c5f1d51669d242387d5fcd19a11"
server_key_version: "encryptionkey@0"
I20260807 22:35:50.987795 5420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:50.988623 5439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:50.988812 5420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:50.988919 5420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "8b06e445ec1c4b1496ae351e79d8f793"
format_stamp: "Formatted at 2026-08-07 22:35:50 on dist-test-slave-7kzw"
server_key: "498ca1300c5d8fa00f4d7bd0041ece5f"
server_key_iv: "98c55c5f1d51669d242387d5fcd19a11"
server_key_version: "encryptionkey@0"
I20260807 22:35:50.989004 5420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:51.006775 5420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:51.009313 5420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:51.009476 5420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:51.009764 5420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:51.010097 5420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:51.010154 5420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.010242 5420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:51.010278 5420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.016414 5420 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:38485
I20260807 22:35:51.016472 5606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:38485 every 8 connection(s)
I20260807 22:35:51.016782 5420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:51.022785 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 5420
I20260807 22:35:51.022869 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:51.023070 29003 external_mini_cluster.cc:1428] Setting key 63a68b1a2677a58a256751fa2e34e475
I20260807 22:35:51.026835 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.017509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:53503 (local address 127.28.82.254:36739)
0807 22:35:51.017556 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.017558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.017772 (+ 214us) server_negotiation.cc:407] Connection header received
0807 22:35:51.018318 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.018320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.018334 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.018389 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.019385 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.019899 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.020901 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.021116 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.021198 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.023524 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.023525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.023525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.023533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.025307 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.025717 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.025718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.025719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.025794 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.026116 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.026117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.026117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.026234 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.026289 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.026658 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.026765 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 22:35:51.027069 5614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.017475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:53503)
0807 22:35:51.017626 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.017670 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.018195 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.018457 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.018483 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.019071 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.019265 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.019270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.020004 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.020005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.020802 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.020805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.021206 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.021207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.021344 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.021854 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.021863 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.023437 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.025397 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.025404 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.025406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.025631 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.025877 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.025878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.025879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.026042 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.026303 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.026310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.026506 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.026636 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.026823 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260807 22:35:51.027623 5610 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36739
I20260807 22:35:51.027746 5610 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:51.027933 5610 heartbeater.cc:511] Master 127.28.82.254:36739 requested a full tablet report, sending...
I20260807 22:35:51.028481 5332 ts_manager.cc:194] Registered new tserver with Master: 8b06e445ec1c4b1496ae351e79d8f793 (127.28.82.193:38485)
I20260807 22:35:51.029179 5332 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:53503
WARNING: no policy specified for oryx/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab.
Entry for principal oryx/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab.
I20260807 22:35:51.052999 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36739
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab
--principal=oryx/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:51.135217 5619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:51.135476 5619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:51.135515 5619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:51.136638 5619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:51.136713 5619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:51.136756 5619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:51.138201 5619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab
--principal=oryx/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:51.138486 5619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:51.138744 5619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:51.146692 5627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.146704 5625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.146979 5630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:51.147346 5619 server_base.cc:1034] running on GCE node
I20260807 22:35:51.147552 5619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:51.147805 5619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:51.148983 5619 hybrid_clock.cc:648] HybridClock initialized: now 1786142151148968 us; error 33 us; skew 500 ppm
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:51.151850 5619 init.cc:377] Logged in from keytab as oryx/127.28.82.194@KRBTEST.COM (short username oryx)
I20260807 22:35:51.152380 5619 webserver.cc:460] Webserver started at http://127.28.82.194:46705/ using document root <none> and password file <none>
I20260807 22:35:51.152616 5619 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:51.152686 5619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:51.152846 5619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:51.153910 5619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "1038c95d4b5445c88e918ea5fcf5ffdc"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "885c24f71047fe9be82a8d759e6cba33"
server_key_iv: "1c857bfae612450add03cabb2dbb27b6"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.154299 5619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "1038c95d4b5445c88e918ea5fcf5ffdc"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "885c24f71047fe9be82a8d759e6cba33"
server_key_iv: "1c857bfae612450add03cabb2dbb27b6"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.155730 5619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:51.156525 5638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.156769 5619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:51.156880 5619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "1038c95d4b5445c88e918ea5fcf5ffdc"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "885c24f71047fe9be82a8d759e6cba33"
server_key_iv: "1c857bfae612450add03cabb2dbb27b6"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.156965 5619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:51.188256 5619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:51.191057 5619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:51.191236 5619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:51.191516 5619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:51.191931 5619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:51.191982 5619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.192018 5619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:51.192050 5619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.200284 5619 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:37031
I20260807 22:35:51.200361 5802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:37031 every 8 connection(s)
I20260807 22:35:51.200742 5619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:51.208884 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 5619
I20260807 22:35:51.208987 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:51.209219 29003 external_mini_cluster.cc:1428] Setting key a2760edd3a6dd4b1c200a75fb4469019
I20260807 22:35:51.213303 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.201509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:55147 (local address 127.28.82.254:36739)
0807 22:35:51.201630 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.201632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.201849 (+ 217us) server_negotiation.cc:407] Connection header received
0807 22:35:51.202733 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.202734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.202745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.202801 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.204188 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.204727 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.206354 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.206611 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.206674 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.209447 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.209448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.209448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.209455 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.211623 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.212067 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.212068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.212069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.212137 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.212452 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.212454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.212454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.212625 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.212712 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.213051 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.213158 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:35:51.213451 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.201483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:55147)
0807 22:35:51.201710 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.201764 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.202596 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.202875 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.202879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.203763 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.204060 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.204066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.204826 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.204827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.206258 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.206261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.206707 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.206709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.206869 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.207417 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.207429 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.209259 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.211723 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.211731 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.211733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.211956 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.212224 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.212225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.212225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.212368 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.212711 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.212717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.212886 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.213010 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.213188 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":121,"threads_started":1}
I20260807 22:35:51.214039 5803 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36739
I20260807 22:35:51.214160 5803 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:51.214342 5803 heartbeater.cc:511] Master 127.28.82.254:36739 requested a full tablet report, sending...
I20260807 22:35:51.214751 5332 ts_manager.cc:194] Registered new tserver with Master: 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031)
I20260807 22:35:51.215149 5332 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:55147
WARNING: no policy specified for oryx/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab.
Entry for principal oryx/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab.
I20260807 22:35:51.241479 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36739
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab
--principal=oryx/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:51.323832 5810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:51.324072 5810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:51.324113 5810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:51.325258 5810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:51.325332 5810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:51.325377 5810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:51.326927 5810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab
--principal=oryx/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:51.327193 5810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:51.327450 5810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:51.332449 5815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.332449 5817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.332451 5820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:51.332921 5810 server_base.cc:1034] running on GCE node
I20260807 22:35:51.333164 5810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:51.333415 5810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:51.334528 5810 hybrid_clock.cc:648] HybridClock initialized: now 1786142151334503 us; error 40 us; skew 500 ppm
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:51.337153 5810 init.cc:377] Logged in from keytab as oryx/127.28.82.195@KRBTEST.COM (short username oryx)
I20260807 22:35:51.337633 5810 webserver.cc:460] Webserver started at http://127.28.82.195:37111/ using document root <none> and password file <none>
I20260807 22:35:51.337863 5810 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:51.337934 5810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:51.338079 5810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:51.339174 5810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b890136b485ed559bc9550bb5008b9a5"
server_key_iv: "13bf81c93da134449aafa2d44fd35525"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.339552 5810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b890136b485ed559bc9550bb5008b9a5"
server_key_iv: "13bf81c93da134449aafa2d44fd35525"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.340883 5810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:51.341642 5831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.341852 5810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:51.341964 5810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b890136b485ed559bc9550bb5008b9a5"
server_key_iv: "13bf81c93da134449aafa2d44fd35525"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.342038 5810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:51.356271 5810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:51.358340 5810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:51.358552 5810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:51.358774 5810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:51.359086 5810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:51.359151 5810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.359195 5810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:51.359222 5810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.365284 5810 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:38963
I20260807 22:35:51.365338 5986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:38963 every 8 connection(s)
I20260807 22:35:51.365634 5810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:51.366453 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 5810
I20260807 22:35:51.366569 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:51.366779 29003 external_mini_cluster.cc:1428] Setting key 92ba39416274ff7396bf7a917a22938f
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:51.374853 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.366356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:36303 (local address 127.28.82.254:36739)
0807 22:35:51.366429 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.366431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.366654 (+ 223us) server_negotiation.cc:407] Connection header received
0807 22:35:51.367339 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.367340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.367353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.367399 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.368330 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.368834 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.369854 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.370055 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.370145 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.372296 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.372298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.372298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.372304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.373656 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.374016 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.374016 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.374017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.374069 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.374282 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.374282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.374283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.374382 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.374432 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.374730 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.374791 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:35:51.375010 5992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.366298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:36303)
0807 22:35:51.366495 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.366546 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.367211 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.367463 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.367467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.368045 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.368228 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.368232 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.368940 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.368941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.369756 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.369759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.370142 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.370143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.370279 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.370766 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.370775 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.372214 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.373729 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.373735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.373737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.373951 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.374126 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.374127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.374127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.374227 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.374444 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.374448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.374606 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.374698 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.374815 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260807 22:35:51.375505 5989 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36739
I20260807 22:35:51.375625 5989 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:51.375957 5989 heartbeater.cc:511] Master 127.28.82.254:36739 requested a full tablet report, sending...
I20260807 22:35:51.376332 5332 ts_manager.cc:194] Registered new tserver with Master: e0d66649e0dd4ce1bd1eea96ddf2c74c (127.28.82.195:38963)
I20260807 22:35:51.376667 5332 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:36303
I20260807 22:35:51.378365 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.82.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.253.keytab.
Entry for principal oryx/127.28.82.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.253.keytab.
I20260807 22:35:51.405821 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.253:35269
--webserver_interface=127.28.82.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.253.keytab
--principal=oryx/127.28.82.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:36739,127.28.82.253:35269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:51.489638 5997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:51.489876 5997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:51.489916 5997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:51.491125 5997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:51.491168 5997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:51.491183 5997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:51.491196 5997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:51.491209 5997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:51.492784 5997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:36739,127.28.82.253:35269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.253.keytab
--principal=oryx/127.28.82.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.253:35269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.82.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
W20260807 22:35:51.492894 5997 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.28.82.254:36739,127.28.82.253:35269'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 22:35:51.493005 5997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:51.493217 5997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:51.498147 6008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:51.498250 5997 server_base.cc:1034] running on GCE node
W20260807 22:35:51.498108 6004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.498122 6005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:51.498667 5997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:51.499023 5997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:51.500192 5997 hybrid_clock.cc:648] HybridClock initialized: now 1786142151500167 us; error 40 us; skew 500 ppm
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:51.503111 5997 init.cc:377] Logged in from keytab as oryx/127.28.82.253@KRBTEST.COM (short username oryx)
I20260807 22:35:51.503584 5997 webserver.cc:460] Webserver started at http://127.28.82.253:36931/ using document root <none> and password file <none>
I20260807 22:35:51.503793 5997 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:51.503839 5997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:51.503926 5997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:51.504993 5997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data/instance:
uuid: "20c8d9158ef54b8daf236ea65cafaf55"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b1aa6c09dbd183aa2acfe1521c885d3c"
server_key_iv: "139dbc72ee8e1fc15e316d0dcf1c9a0e"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.505311 5997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal/instance:
uuid: "20c8d9158ef54b8daf236ea65cafaf55"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b1aa6c09dbd183aa2acfe1521c885d3c"
server_key_iv: "139dbc72ee8e1fc15e316d0dcf1c9a0e"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.506635 5997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:51.507426 6016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.507654 5997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:51.507781 5997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
uuid: "20c8d9158ef54b8daf236ea65cafaf55"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b1aa6c09dbd183aa2acfe1521c885d3c"
server_key_iv: "139dbc72ee8e1fc15e316d0dcf1c9a0e"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.507872 5997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:51.522169 5997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:51.524153 5997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:51.524317 5997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:51.528371 5997 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.253:35269
I20260807 22:35:51.528419 6089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.253:35269 every 8 connection(s)
I20260807 22:35:51.528790 5997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
I20260807 22:35:51.529805 6090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:51.530606 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 5997
I20260807 22:35:51.531847 6090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.253@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.82.253@KRBTEST.COM
I20260807 22:35:51.541678 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.532728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.28.82.254:36739)
0807 22:35:51.532773 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.532776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.532935 (+ 159us) server_negotiation.cc:407] Connection header received
0807 22:35:51.533015 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.533016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.533027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.533093 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.533553 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.534298 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.535595 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.535889 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.535955 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.538253 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.538255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.538255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.538264 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.539858 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.540364 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.540364 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.540365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.540566 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.540843 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.540844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.540845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.541068 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.541147 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.541474 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.541565 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:35:51.541831 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.532650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:41236)
0807 22:35:51.532832 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.532870 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.532950 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.533123 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.533125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.533369 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.533458 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.533462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.534399 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.534399 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.535504 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.535506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.535976 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.535977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.536051 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.536499 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.536508 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.538161 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.539956 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.539961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.539963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.540222 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.540652 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.540653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.540653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.540758 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.541160 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.541165 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.541312 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.541440 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.541600 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260807 22:35:51.541956 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.531273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.28.82.253:35269)
0807 22:35:51.531465 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.531475 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.531519 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:35:51.532072 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.532081 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.532145 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.532302 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.533066 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.534280 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.535369 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.535780 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.535892 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.538752 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.538760 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.538760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.538768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.540395 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.540883 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.540886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.540887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.540947 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.541152 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.541153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.541153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.541317 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.541474 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.541582 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.541677 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260807 22:35:51.542428 6090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.253@KRBTEST.COM for oryx/127.28.82.253@KRBTEST.COM
I20260807 22:35:51.548640 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.542719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.0.0.1:50010)
0807 22:35:51.542788 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.542798 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.542837 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.542936 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.542937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.543044 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.543087 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.543088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.543657 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.543657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.544100 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.544101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.544530 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.544530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.544581 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.544927 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.544931 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.546184 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.547601 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.547602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.547602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.547802 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.548016 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.548017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.548017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.548074 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.548322 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.548322 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.548359 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.548427 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.548522 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 22:35:51.548656 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.542788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.28.82.253:35269)
0807 22:35:51.542820 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.542822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.542830 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:51.542888 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.542888 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.542895 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.542934 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.543153 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.543591 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.544194 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.544458 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.544520 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.546283 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.546284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.546284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.546291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.547504 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.547891 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.547892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.547892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.547942 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.548146 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.548147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.548147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.548251 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.548328 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.548452 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.548571 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":15}
I20260807 22:35:51.549947 6090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Bootstrap starting.
I20260807 22:35:51.550897 6090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:51.551141 6090 log.cc:824] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:51.551817 6090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: No bootstrap required, opened a new log
I20260807 22:35:51.553205 6090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } }
I20260807 22:35:51.553337 6090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:51.553360 6090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c8d9158ef54b8daf236ea65cafaf55, State: Initialized, Role: FOLLOWER
I20260807 22:35:51.553444 6090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } }
I20260807 22:35:51.553674 6099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } } }
I20260807 22:35:51.553743 6090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:51.553771 6099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:35:51.555573 6115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:35:51.555639 6115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:35:51.557246 5997 master_runner.cc:428] Detected that this master 20c8d9158ef54b8daf236ea65cafaf55 is joining an existing cluster
I20260807 22:35:51.557317 5997 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.82.253:35269
I20260807 22:35:51.565215 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.558464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.28.82.253:35269)
0807 22:35:51.558528 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.558529 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.558988 (+ 459us) server_negotiation.cc:407] Connection header received
0807 22:35:51.559057 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.559058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.559067 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.559134 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.559481 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.559935 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.560670 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.560845 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.560918 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.562019 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.562020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.562020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.562026 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.563644 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.564068 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.564070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.564070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.564144 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.564619 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.564620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.564620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.564788 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.564837 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.565018 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.565129 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:35:51.565580 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.558320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.0.0.1:50024)
0807 22:35:51.558921 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.558930 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.559002 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.559159 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.559160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.559326 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.559398 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.559399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.560015 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.560016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.560579 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.560581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.560941 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.560942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.561053 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.561527 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.561532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.561928 (+ 396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.563738 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.563739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.563747 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.563983 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.564218 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.564219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.564220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.564291 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.564882 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.564882 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.564917 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.565394 (+ 477us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.565485 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":86,"threads_started":1}
I20260807 22:35:51.568130 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.558369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.28.82.254:36739)
0807 22:35:51.558436 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.558438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.561190 (+ 2752us) server_negotiation.cc:407] Connection header received
0807 22:35:51.561244 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.561244 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.561268 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.561310 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.561673 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.562428 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.563794 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.563998 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.564081 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.565434 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.565436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.565436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.565443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.566972 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.567339 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.567340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.567341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.567399 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.567607 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.567608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.567608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.567728 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.567807 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.567972 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.568042 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:35:51.568154 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.558768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:41246)
0807 22:35:51.561105 (+ 2337us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.561116 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.561181 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.561334 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.561335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.561543 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.561584 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.561586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.562788 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.562789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.563698 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.563700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.564219 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.564220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.564275 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.564946 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.564950 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.565275 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.567055 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.567056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.567057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.567232 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.567479 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.567480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.567481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.567540 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.567800 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.567801 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.567865 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.567942 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.568040 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2310,"thread_start_us":89,"threads_started":1}
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.254@KRBTEST.COM for oryx/127.28.82.253@KRBTEST.COM
I20260807 22:35:51.577010 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.570265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50030 (local address 127.28.82.253:35269)
0807 22:35:51.570305 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.570307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.570439 (+ 132us) server_negotiation.cc:407] Connection header received
0807 22:35:51.570550 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.570551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.570561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.570616 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.571021 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.571468 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.572248 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.572454 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.572530 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.574503 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.574504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.574505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.574512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.575807 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.576212 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.576213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.576213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.576285 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.576525 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.576526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.576526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.576620 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.576698 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.576899 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.576956 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:51.577098 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.570205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.0.0.1:50030)
0807 22:35:51.570328 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.570369 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.570464 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.570622 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.570625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.570896 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.570938 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.570939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.571554 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.571555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.572160 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.572162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.572531 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.572532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.572609 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.573073 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.573077 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.574393 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.575907 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.575915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.575917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.576118 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.576372 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.576373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.576374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.576452 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.576713 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.576720 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.576782 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.576874 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.576995 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260807 22:35:51.577725 5332 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.82.253:35269: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55"
member_type: NON_VOTER
last_known_addr {
host: "127.28.82.253"
port: 35269
}
attrs {
promote: true
}
}
dest_uuid: "21af200769d44c058aa2fb4efd2fd9a2"
cas_config_opid_index: -1
I20260807 22:35:51.578045 5332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } }
I20260807 22:35:51.578536 6129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } }
I20260807 22:35:51.578894 6050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:51.579556 6129 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.82.253:35269
I20260807 22:35:51.579715 6129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } }
I20260807 22:35:51.579823 6129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:51.579828 6050 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Refusing update from remote peer 21af200769d44c058aa2fb4efd2fd9a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 22:35:51.579859 5997 master.cc:537] Master@127.28.82.253:35269 shutting down...
I20260807 22:35:51.580325 6130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:35:51.581941 5302 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 22:35:51.582362 5302 consensus_peers.cc:487] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 -> Peer 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253:35269): Couldn't send request to peer 20c8d9158ef54b8daf236ea65cafaf55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 22:35:51.582942 5997 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:51.583036 5997 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:51.583086 5997 tablet_replica.cc:330] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: stopping tablet replica
I20260807 22:35:51.596208 5997 master.cc:552] Master@127.28.82.253:35269 shutdown complete.
I20260807 22:35:51.596279 5997 master_runner.cc:305] Clearing existing system tablet
I20260807 22:35:51.596406 5997 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:51.596470 5997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:51.597092 5997 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260807 22:35:51.597596 6134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.597718 5997 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:51.597779 5997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.597826 5997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
uuid: "20c8d9158ef54b8daf236ea65cafaf55"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b1aa6c09dbd183aa2acfe1521c885d3c"
server_key_iv: "139dbc72ee8e1fc15e316d0dcf1c9a0e"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.598438 5997 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:51.599773 5997 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 22:35:51.599853 5997 log.cc:1196] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 22:35:51.600049 5997 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Deleting consensus metadata
I20260807 22:35:51.600175 5997 master_runner.cc:315] Copying system tablet from 127.28.82.254:36739
I20260807 22:35:51.600809 5997 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: tablet copy: Beginning tablet copy session from remote peer at address 127.28.82.254:36739
I20260807 22:35:51.606950 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.601163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.28.82.254:36739)
0807 22:35:51.601212 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.601214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.601337 (+ 123us) server_negotiation.cc:407] Connection header received
0807 22:35:51.601427 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.601428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.601439 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.601501 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.601831 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.602318 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.603124 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.603334 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.603408 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.604370 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.604371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.604372 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.604379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.605676 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.606043 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.606044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.606045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.606109 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.606326 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.606327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.606327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.606442 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.606605 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.606785 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.606859 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:35:51.606966 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.601110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:41248)
0807 22:35:51.601268 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.601277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.601364 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.601507 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.601508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.601687 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.601749 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.601750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.602401 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.602402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.603019 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.603021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.603419 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.603420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.603486 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.603980 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.603985 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.604278 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.605771 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.605772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.605772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.605953 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.606178 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.606179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.606179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.606258 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.606594 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.606595 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.606641 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.606755 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.606860 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
I20260807 22:35:51.607474 5357 tablet_copy_service.cc:140] P 21af200769d44c058aa2fb4efd2fd9a2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 20c8d9158ef54b8daf236ea65cafaf55 ({username='oryx', principal='oryx/127.28.82.253@KRBTEST.COM'} at 127.0.0.1:41248)
I20260807 22:35:51.607583 5357 tablet_copy_service.cc:161] P 21af200769d44c058aa2fb4efd2fd9a2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 20c8d9158ef54b8daf236ea65cafaf55 at {username='oryx', principal='oryx/127.28.82.253@KRBTEST.COM'} at 127.0.0.1:41248: session id = 20c8d9158ef54b8daf236ea65cafaf55-00000000000000000000000000000000
I20260807 22:35:51.608399 5357 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 22:35:51.609010 5997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 22:35:51.610109 5997 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: tablet copy: Starting download of 0 data blocks...
I20260807 22:35:51.610270 5997 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: tablet copy: Starting download of 1 WAL segments...
I20260807 22:35:51.611194 5997 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 22:35:51.612008 5357 tablet_copy_service.cc:342] P 21af200769d44c058aa2fb4efd2fd9a2: Request end of tablet copy session 20c8d9158ef54b8daf236ea65cafaf55-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.82.253@KRBTEST.COM'} at 127.0.0.1:41248
I20260807 22:35:51.612075 5357 tablet_copy_service.cc:433] P 21af200769d44c058aa2fb4efd2fd9a2: ending tablet copy session 20c8d9158ef54b8daf236ea65cafaf55-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 20c8d9158ef54b8daf236ea65cafaf55
I20260807 22:35:51.614611 5997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:51.615808 6151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.615883 6152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.615839 6154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:51.616387 5997 server_base.cc:1034] running on GCE node
I20260807 22:35:51.616495 5997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:51.616633 5997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:51.617770 5997 hybrid_clock.cc:648] HybridClock initialized: now 1786142151617767 us; error 30 us; skew 500 ppm
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:51.619225 5997 init.cc:377] Logged in from keytab as oryx/127.28.82.253@KRBTEST.COM (short username oryx)
I20260807 22:35:51.619515 5997 webserver.cc:460] Webserver started at http://127.28.82.253:43569/ using document root <none> and password file <none>
I20260807 22:35:51.619630 5997 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:51.619657 5997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:51.620396 5997 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:51.621071 6164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.621210 5997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:51.621265 5997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
uuid: "20c8d9158ef54b8daf236ea65cafaf55"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b1aa6c09dbd183aa2acfe1521c885d3c"
server_key_iv: "139dbc72ee8e1fc15e316d0dcf1c9a0e"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.621328 5997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:51.644963 5997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:51.646948 5997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:51.650151 5997 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.253:35269
I20260807 22:35:51.650170 6240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.253:35269 every 8 connection(s)
I20260807 22:35:51.651087 5997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
I20260807 22:35:51.651958 6241 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:35:51.652455 6241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Bootstrap starting.
I20260807 22:35:51.654767 6241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:35:51.654989 6241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: Bootstrap complete.
I20260807 22:35:51.655205 6241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } }
I20260807 22:35:51.655275 6241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20c8d9158ef54b8daf236ea65cafaf55, State: Initialized, Role: LEARNER
I20260807 22:35:51.655303 6241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } }
I20260807 22:35:51.655491 6241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:51.655504 6244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } }
I20260807 22:35:51.655628 6244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:35:51.656935 6258 catalog_manager.cc:1259] Loaded cluster ID: 951f417b1e7f44d5b8886c82cc9e9e6e
I20260807 22:35:51.656986 6258 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: loading cluster ID for follower catalog manager: success
I20260807 22:35:51.658104 6258 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: acquiring CA information for follower catalog manager: success
I20260807 22:35:51.658391 6258 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:35:51.700963 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.694194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50052 (local address 127.28.82.253:35269)
0807 22:35:51.694350 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.694352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.694383 (+ 31us) server_negotiation.cc:407] Connection header received
0807 22:35:51.694407 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.694408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.694418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.694544 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.695164 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.695819 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.696593 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.696789 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.696884 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.698203 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.698205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.698205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.698214 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.699640 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.700081 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.700082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.700082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.700152 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.700375 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.700376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.700377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.700498 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.700616 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.700749 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.700843 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for oryx/127.28.82.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.252.keytab.
Entry for principal oryx/127.28.82.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.252.keytab.
I20260807 22:35:51.731403 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.252:44363
--webserver_interface=127.28.82.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.252.keytab
--principal=oryx/127.28.82.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:36739,127.28.82.253:35269,127.28.82.252:44363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:51.818184 6265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:51.818398 6265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:51.818421 6265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:51.819639 6265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:51.819691 6265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:51.819706 6265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:51.819741 6265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:51.819764 6265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:51.821352 6265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:36739,127.28.82.253:35269,127.28.82.252:44363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.252.keytab
--principal=oryx/127.28.82.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.252:44363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.82.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:51.821545 6265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:51.821725 6265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:51.826743 6275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.826759 6277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:51.826759 6279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:51.827076 6265 server_base.cc:1034] running on GCE node
I20260807 22:35:51.827349 6265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:51.827642 6265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:51.828835 6265 hybrid_clock.cc:648] HybridClock initialized: now 1786142151828815 us; error 49 us; skew 500 ppm
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:51.831470 6265 init.cc:377] Logged in from keytab as oryx/127.28.82.252@KRBTEST.COM (short username oryx)
I20260807 22:35:51.831939 6265 webserver.cc:460] Webserver started at http://127.28.82.252:44239/ using document root <none> and password file <none>
I20260807 22:35:51.832171 6265 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:51.832242 6265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:51.832399 6265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:51.833459 6265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data/instance:
uuid: "f6b627b3aca44300b46e08e129409427"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "440252f475f13ae23273dc54e4bbaaad"
server_key_iv: "f3f99109f6b22765f234c9ef06bb72a4"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.833843 6265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal/instance:
uuid: "f6b627b3aca44300b46e08e129409427"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "440252f475f13ae23273dc54e4bbaaad"
server_key_iv: "f3f99109f6b22765f234c9ef06bb72a4"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.835310 6265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:51.836102 6288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.836280 6265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:51.836391 6265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
uuid: "f6b627b3aca44300b46e08e129409427"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "440252f475f13ae23273dc54e4bbaaad"
server_key_iv: "f3f99109f6b22765f234c9ef06bb72a4"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.836467 6265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:51.852773 6265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:51.854755 6265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:51.854923 6265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:51.859644 6265 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.252:44363
I20260807 22:35:51.859696 6356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.252:44363 every 8 connection(s)
I20260807 22:35:51.860085 6265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
I20260807 22:35:51.861177 6357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:51.862823 6357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:35:51.867637 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 6265
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.252@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:51.873926 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.863565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.28.82.254:36739)
0807 22:35:51.863618 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.863620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.863905 (+ 285us) server_negotiation.cc:407] Connection header received
0807 22:35:51.864398 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.864400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.864414 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.864471 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.865075 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.865709 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.866705 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.866912 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.866988 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.869704 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.869705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.869706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.869714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.872068 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.872487 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.872489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.872490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.872561 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.872990 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.872991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.872992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.873183 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.873238 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.873677 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.873798 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:35:51.874253 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.863480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:41264)
0807 22:35:51.863784 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.863817 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.864287 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.864540 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.864544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.864787 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.864972 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.864976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.865805 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.865806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.866608 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.866610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.867025 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.867026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.867145 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.867637 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.867647 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.869594 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.872153 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.872160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.872162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.872406 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.872695 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.872696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.872697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.872888 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.873310 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.873317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.873488 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.873613 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.873796 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":240,"threads_started":1}
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.82.252@KRBTEST.COM
I20260807 22:35:51.874924 6357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:35:51.878008 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.868195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.28.82.252:44363)
0807 22:35:51.868432 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.868437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.868458 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:35:51.868481 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.868484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.868504 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.868610 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.869450 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.870695 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.871686 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.872001 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.872134 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.875117 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.875124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.875124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.875133 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.876743 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.877137 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.877140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.877141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.877212 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.877423 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.877424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.877424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.877567 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.877743 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.877799 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.877894 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":80,"threads_started":1}
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.252@KRBTEST.COM for oryx/127.28.82.253@KRBTEST.COM
I20260807 22:35:51.881336 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.875196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.0.0.1:50058)
0807 22:35:51.875238 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.875253 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.875288 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.875451 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.875452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.875594 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.875636 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.875637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.876298 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.876299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.876800 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.876801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.877198 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.877199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.877274 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.877667 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.877678 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.878985 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.880339 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.880340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.880341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.880507 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.880715 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.880715 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.880716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.880774 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.881040 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.881041 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.881081 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.881163 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.881251 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 22:35:51.881387 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.875264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.28.82.253:35269)
0807 22:35:51.875310 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.875312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.875333 (+ 21us) server_negotiation.cc:407] Connection header received
0807 22:35:51.875360 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.875361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.875373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.875446 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.875724 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.876213 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.876884 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.877105 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.877197 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.879070 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.879072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.879072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.879080 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.880243 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.880579 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.880580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.880580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.880637 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.880844 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.880845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.880846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.880960 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.881037 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.881209 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.881294 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:51.881798 6357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142151, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.252@KRBTEST.COM for oryx/127.28.82.252@KRBTEST.COM
I20260807 22:35:51.887876 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.882065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.0.0.1:55868)
0807 22:35:51.882112 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.882122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.882160 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.882260 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.882260 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.882363 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.882388 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.882389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.882982 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.882983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.883419 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.883420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.883735 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.883736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.883772 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.884078 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.884082 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.885123 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.886746 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.886747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.886747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.886887 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.887124 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.887125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.887125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.887216 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.887541 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.887542 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.887589 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.887668 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.887772 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 22:35:51.888152 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.882110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.28.82.252:44363)
0807 22:35:51.882160 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.882161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.882170 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:35:51.882208 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.882208 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.882216 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.882258 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.882445 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.882917 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.883486 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.883675 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.883724 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.885207 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.885209 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.885209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.885215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.886653 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.886967 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.886969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.886969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.887051 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.887313 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.887314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.887314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.887473 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.887550 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.887726 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.888086 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":32}
I20260807 22:35:51.889505 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Bootstrap starting.
I20260807 22:35:51.890416 6357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:51.890772 6357 log.cc:824] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:51.891513 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: No bootstrap required, opened a new log
I20260807 22:35:51.893033 6357 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } }
I20260807 22:35:51.893160 6357 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:51.893184 6357 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b627b3aca44300b46e08e129409427, State: Initialized, Role: FOLLOWER
I20260807 22:35:51.893332 6357 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } }
I20260807 22:35:51.893625 6357 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:51.893599 6366 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } } }
I20260807 22:35:51.893704 6366 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:35:51.895476 6384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:35:51.895534 6384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:35:51.897706 6265 master_runner.cc:428] Detected that this master f6b627b3aca44300b46e08e129409427 is joining an existing cluster
I20260807 22:35:51.897749 6265 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.82.252:44363
I20260807 22:35:51.906813 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.899494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.0.0.1:55872)
0807 22:35:51.899742 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.899751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.899815 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.899968 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.899969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.900213 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.900275 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.900276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.901084 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.901084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.901930 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.901932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.902711 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.902712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.902794 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.903193 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.903198 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.903526 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.905188 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.905189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.905189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.905363 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.905570 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.905571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.905571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.906164 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.906545 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.906546 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.906591 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.906646 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.906738 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":172,"threads_started":1}
I20260807 22:35:51.906908 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.899532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.28.82.252:44363)
0807 22:35:51.899576 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.899577 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.899795 (+ 218us) server_negotiation.cc:407] Connection header received
0807 22:35:51.899875 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.899876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.899883 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.899906 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.900351 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.901006 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.902009 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.902600 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.902646 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.903615 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.903617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.903617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.903624 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.905107 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.905434 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.905435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.905435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.905501 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.906241 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.906242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.906242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.906439 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.906551 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.906693 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.906801 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":17}
I20260807 22:35:51.921599 6366 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:51.921730 6366 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } }
I20260807 22:35:51.922233 6366 leader_election.cc:288] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739), 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253:35269)
I20260807 22:35:51.922494 5347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6b627b3aca44300b46e08e129409427" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21af200769d44c058aa2fb4efd2fd9a2" is_pre_election: true
I20260807 22:35:51.922518 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6b627b3aca44300b46e08e129409427" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c8d9158ef54b8daf236ea65cafaf55" is_pre_election: true
I20260807 22:35:51.922636 6203 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 LEARNER]: Handling vote request from an unknown peer f6b627b3aca44300b46e08e129409427
I20260807 22:35:51.922712 6203 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate f6b627b3aca44300b46e08e129409427 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 22:35:51.922928 6290 leader_election.cc:302] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6b627b3aca44300b46e08e129409427; no voters: 20c8d9158ef54b8daf236ea65cafaf55, 21af200769d44c058aa2fb4efd2fd9a2
I20260807 22:35:51.923089 6366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:51.924213 6366 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 22:35:51.926615 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.898807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.28.82.253:35269)
0807 22:35:51.898853 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.898854 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.899208 (+ 354us) server_negotiation.cc:407] Connection header received
0807 22:35:51.899277 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.899278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.899289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.899345 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.899817 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.900601 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.901663 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.901952 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.901992 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.923383 (+ 21391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.923385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.923385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.923395 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.925300 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.925671 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.925672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.925673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.925736 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.926018 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.926018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.926019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.926168 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.926252 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.926400 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.926513 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:51.926640 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.898800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.0.0.1:50066)
0807 22:35:51.899115 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.899129 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.899217 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.899360 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.899361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.899576 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.899711 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.899714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.900687 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.900688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.901571 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.901574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.902076 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.902077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.902140 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.902609 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.902615 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.923285 (+ 20670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.925405 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.925406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.925407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.925592 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.925805 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.925806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.925806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.925921 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.926245 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.926246 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.926295 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.926369 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.926526 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":2,"thread_start_us":552,"threads_started":1}
I20260807 22:35:51.927352 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.898778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41268 (local address 127.28.82.254:36739)
0807 22:35:51.898820 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.898822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.899787 (+ 965us) server_negotiation.cc:407] Connection header received
0807 22:35:51.899881 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.899882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.899893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.899923 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.900394 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.901148 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.902214 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.902544 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.902602 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.923897 (+ 21295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.923898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.923898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.923907 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.925651 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.926001 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.926002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.926002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.926067 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.926294 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.926295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.926296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.926437 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.926539 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.927174 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.927253 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:35:51.927368 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.898904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:41268)
0807 22:35:51.899724 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.899734 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.899812 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.899984 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.899984 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.900231 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.900297 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.900299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.901248 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.901249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.902110 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.902113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.902681 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.902682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.902750 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.923410 (+ 20660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.923415 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.923787 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.925730 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.925731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.925731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.925918 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.926140 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.926141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.926141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.926204 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.926568 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.926569 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.926622 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.927135 (+ 513us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.927252 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":1,"thread_start_us":130,"threads_started":1}
I20260807 22:35:51.929386 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 5420
I20260807 22:35:51.948699 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:38485
--local_ip_for_outbound_sockets=127.28.82.193
--tserver_master_addrs=127.28.82.254:36739
--webserver_port=41303
--webserver_interface=127.28.82.193
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab
--principal=oryx/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269,127.28.82.252:44363 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:35:51 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142150, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.254@KRBTEST.COM for oryx/127.28.82.252@KRBTEST.COM
I20260807 22:35:51.952796 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.944300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.0.0.1:55888)
0807 22:35:51.944357 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.944368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.944406 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.944622 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.944624 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.944818 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.944873 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.944875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.945866 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.945867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.946771 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.946774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.947328 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.947329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.947415 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.947930 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.947936 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.949576 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.951385 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.951386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.951386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.951662 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.952006 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.952007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.952007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.952088 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.952430 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.952432 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.952486 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.952568 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.952687 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 22:35:51.952845 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.944359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.28.82.252:44363)
0807 22:35:51.944398 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.944400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.944410 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:51.944510 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.944512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.944523 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.944573 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.944955 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.945775 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.946864 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.947209 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.947330 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.949669 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.949670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.949670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.949678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.951291 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.951757 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.951758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.951759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.951826 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.952196 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.952197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.952197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.952350 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.952422 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.952616 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.952706 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 22:35:51.953547 5332 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.82.252:44363: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f6b627b3aca44300b46e08e129409427"
member_type: NON_VOTER
last_known_addr {
host: "127.28.82.252"
port: 44363
}
attrs {
promote: true
}
}
dest_uuid: "21af200769d44c058aa2fb4efd2fd9a2"
cas_config_opid_index: 5
I20260807 22:35:51.953804 5332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } }
I20260807 22:35:51.954175 6130 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f6b627b3aca44300b46e08e129409427 (127.28.82.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.954512 6327 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Refusing update from remote peer 21af200769d44c058aa2fb4efd2fd9a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 22:35:51.955025 6128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:51.955242 6130 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.82.252:44363
I20260807 22:35:51.955318 6130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.955379 6130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:51.955675 6265 master.cc:537] Master@127.28.82.252:44363 shutting down...
I20260807 22:35:51.956625 6327 raft_consensus.cc:858] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } }
W20260807 22:35:51.956712 6327 raft_consensus.cc:1565] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } }
I20260807 22:35:51.957333 6327 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } }
I20260807 22:35:51.959607 6366 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21af200769d44c058aa2fb4efd2fd9a2. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } }
I20260807 22:35:51.959702 6366 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 22:35:51.959740 6366 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } }
I20260807 22:35:51.959790 6366 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 22:35:51.959823 6366 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } }
I20260807 22:35:51.959867 6366 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 22:35:51.960392 6129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.960486 6129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:51.960776 6395 mvcc.cc:205] Tried to move back new op lower bound from 7316038249909665792 to 7316038249717161984. Current Snapshot: MvccSnapshot[applied={T|T < 7316038249726111744}]
W20260807 22:35:51.961374 5304 consensus_peers.cc:487] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 -> Peer f6b627b3aca44300b46e08e129409427 (127.28.82.252:44363): Couldn't send request to peer f6b627b3aca44300b46e08e129409427. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 22:35:51.961603 6265 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 22:35:51.961697 6265 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 22:35:51.961735 6265 tablet_replica.cc:330] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: stopping tablet replica
I20260807 22:35:51.964468 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.954934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50082 (local address 127.28.82.253:35269)
0807 22:35:51.958357 (+ 3423us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.958359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.959124 (+ 765us) server_negotiation.cc:407] Connection header received
0807 22:35:51.959159 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.959160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.959185 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.959257 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:51.959439 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.960205 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.962938 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.963546 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.963618 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.963685 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.964356 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.964400 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3392}
I20260807 22:35:51.964548 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.954824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.0.0.1:50082)
0807 22:35:51.958995 (+ 4171us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.959023 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.959086 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.959323 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.959325 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.959327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:51.959367 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.959368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.961407 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.961408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.962838 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.962841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.963720 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.963721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.963917 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.963941 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.963996 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.964037 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4149}
I20260807 22:35:51.964928 6203 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 LEARNER]: Refusing update from remote peer 21af200769d44c058aa2fb4efd2fd9a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 22:35:51.965144 6128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 22:35:51.965687 6203 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f6b627b3aca44300b46e08e129409427 (127.28.82.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.966830 6408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.966974 6408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:35:51.967041 6244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21af200769d44c058aa2fb4efd2fd9a2. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: NON_VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: true } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.967150 6244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:35:51.967631 6130 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: attempting to promote NON_VOTER 20c8d9158ef54b8daf236ea65cafaf55 to VOTER
I20260807 22:35:51.967816 6130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } }
I20260807 22:35:51.968261 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.968136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.0.0.1:55900)
0807 22:35:51.968185 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.968233 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:35:51.968329 6203 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 LEARNER]: Refusing update from remote peer 21af200769d44c058aa2fb4efd2fd9a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 22:35:51.968492 6128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 22:35:51.968868 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.968789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.0.0.1:55912)
0807 22:35:51.968822 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.968843 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 22:35:51.968932 5304 consensus_peers.cc:487] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 -> Peer f6b627b3aca44300b46e08e129409427 (127.28.82.252:44363): Couldn't send request to peer f6b627b3aca44300b46e08e129409427. Status: Network error: Client connection negotiation failed: client connection to 127.28.82.252:44363: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:35:51.969218 6129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.969357 6203 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.970060 6129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.970129 6129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:51.970184 6129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.970162 6244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:51.970223 6129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:51.970235 6244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:35:51.981958 6265 master.cc:552] Master@127.28.82.252:44363 shutdown complete.
I20260807 22:35:51.982017 6265 master_runner.cc:305] Clearing existing system tablet
I20260807 22:35:51.982162 6265 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:51.982218 6265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:51.983070 6265 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:35:51.983662 6412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:51.983871 6265 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:51.983942 6265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:51.983982 6265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
uuid: "f6b627b3aca44300b46e08e129409427"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "440252f475f13ae23273dc54e4bbaaad"
server_key_iv: "f3f99109f6b22765f234c9ef06bb72a4"
server_key_version: "encryptionkey@0"
I20260807 22:35:51.984674 6265 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:51.986078 6265 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 22:35:51.986162 6265 log.cc:1196] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 22:35:51.986418 6265 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Deleting consensus metadata
I20260807 22:35:51.986581 6265 master_runner.cc:315] Copying system tablet from 127.28.82.254:36739
I20260807 22:35:51.987219 6265 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: tablet copy: Beginning tablet copy session from remote peer at address 127.28.82.254:36739
I20260807 22:35:51.994884 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.987531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:41282)
0807 22:35:51.987702 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:51.987714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:51.987791 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:51.987959 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:51.987960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:51.988187 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:51.988245 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.988247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.989197 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.989199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.990012 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:51.990015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.990538 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.990539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:51.990607 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.991144 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:51.991150 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:51.991595 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:51.993590 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.993592 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.993592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.993821 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.994084 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:51.994085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:51.994086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:51.994172 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:51.994525 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:51.994526 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:51.994575 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 22:35:51.994658 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:35:51.994765 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
I20260807 22:35:51.994935 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:51.987597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.28.82.254:36739)
0807 22:35:51.987647 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:51.987650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:51.987796 (+ 146us) server_negotiation.cc:407] Connection header received
0807 22:35:51.987853 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:51.987854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:51.987870 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:51.987937 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:51.988331 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.989098 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.990108 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:51.990408 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:51.990509 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:51.991710 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:51.991712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:51.991712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:51.991720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:51.993505 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.993932 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.993933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.993934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.993999 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:51.994247 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:51.994248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:51.994248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:51.994385 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:51.994485 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:51.994697 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 22:35:51.994819 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 22:35:51.995558 5357 tablet_copy_service.cc:140] P 21af200769d44c058aa2fb4efd2fd9a2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f6b627b3aca44300b46e08e129409427 ({username='oryx', principal='oryx/127.28.82.252@KRBTEST.COM'} at 127.0.0.1:41282)
I20260807 22:35:51.995636 5357 tablet_copy_service.cc:161] P 21af200769d44c058aa2fb4efd2fd9a2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f6b627b3aca44300b46e08e129409427 at {username='oryx', principal='oryx/127.28.82.252@KRBTEST.COM'} at 127.0.0.1:41282: session id = f6b627b3aca44300b46e08e129409427-00000000000000000000000000000000
I20260807 22:35:51.996049 5357 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 22:35:51.996539 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 22:35:51.997963 6265 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: tablet copy: Starting download of 0 data blocks...
I20260807 22:35:51.998132 6265 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: tablet copy: Starting download of 1 WAL segments...
I20260807 22:35:51.999040 6265 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 22:35:52.000043 5357 tablet_copy_service.cc:342] P 21af200769d44c058aa2fb4efd2fd9a2: Request end of tablet copy session f6b627b3aca44300b46e08e129409427-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.82.252@KRBTEST.COM'} at 127.0.0.1:41282
I20260807 22:35:52.000098 5357 tablet_copy_service.cc:433] P 21af200769d44c058aa2fb4efd2fd9a2: ending tablet copy session f6b627b3aca44300b46e08e129409427-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f6b627b3aca44300b46e08e129409427
I20260807 22:35:52.002691 6265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:52.003948 6425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:52.004482 6265 server_base.cc:1034] running on GCE node
W20260807 22:35:52.003950 6421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:52.006705 6422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:52.006872 6265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:52.007010 6265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:52.008149 6265 hybrid_clock.cc:648] HybridClock initialized: now 1786142152008122 us; error 39 us; skew 500 ppm
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:52.009886 6265 init.cc:377] Logged in from keytab as oryx/127.28.82.252@KRBTEST.COM (short username oryx)
I20260807 22:35:52.010185 6265 webserver.cc:460] Webserver started at http://127.28.82.252:34677/ using document root <none> and password file <none>
I20260807 22:35:52.010313 6265 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:52.010349 6265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:52.011158 6265 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:35:52.011803 6434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.011981 6265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:52.012040 6265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
uuid: "f6b627b3aca44300b46e08e129409427"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "440252f475f13ae23273dc54e4bbaaad"
server_key_iv: "f3f99109f6b22765f234c9ef06bb72a4"
server_key_version: "encryptionkey@0"
I20260807 22:35:52.012104 6265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:52.038154 6265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:52.040237 6265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:52.043578 6265 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.252:44363
I20260807 22:35:52.043622 6500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.252:44363 every 8 connection(s)
I20260807 22:35:52.043927 6265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
I20260807 22:35:52.044979 6502 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:35:52.045629 6502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Bootstrap starting.
I20260807 22:35:52.048249 6502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:35:52.048470 6502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Bootstrap complete.
I20260807 22:35:52.048704 6502 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } }
I20260807 22:35:52.048803 6502 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f6b627b3aca44300b46e08e129409427, State: Initialized, Role: LEARNER
I20260807 22:35:52.048841 6502 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } }
I20260807 22:35:52.049015 6502 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:52.049038 6507 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:52.049253 6507 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:35:52.050575 6526 catalog_manager.cc:1259] Loaded cluster ID: 951f417b1e7f44d5b8886c82cc9e9e6e
I20260807 22:35:52.050683 6526 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: loading cluster ID for follower catalog manager: success
I20260807 22:35:52.051920 6526 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: acquiring CA information for follower catalog manager: success
I20260807 22:35:52.052194 6526 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 22:35:52.053741 6393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:52.053958 6393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:52.054010 6393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:52.055261 6393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:52.055326 6393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:52.055394 6393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:52.057076 6393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab
--principal=oryx/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:38485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=41303
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269,127.28.82.252:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:52.057330 6393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:52.057566 6393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:52.063597 6533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:52.063730 6536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:52.063614 6534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:52.064060 6393 server_base.cc:1034] running on GCE node
I20260807 22:35:52.064275 6393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:52.064512 6393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:52.065661 6393 hybrid_clock.cc:648] HybridClock initialized: now 1786142152065649 us; error 28 us; skew 500 ppm
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:52.068686 6393 init.cc:377] Logged in from keytab as oryx/127.28.82.193@KRBTEST.COM (short username oryx)
I20260807 22:35:52.069113 6393 webserver.cc:460] Webserver started at http://127.28.82.193:41303/ using document root <none> and password file <none>
I20260807 22:35:52.069361 6393 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:52.069449 6393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:52.070960 6393 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:35:52.071583 6548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.071762 6393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:52.071875 6393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "8b06e445ec1c4b1496ae351e79d8f793"
format_stamp: "Formatted at 2026-08-07 22:35:50 on dist-test-slave-7kzw"
server_key: "498ca1300c5d8fa00f4d7bd0041ece5f"
server_key_iv: "98c55c5f1d51669d242387d5fcd19a11"
server_key_version: "encryptionkey@0"
I20260807 22:35:52.072209 6393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:52.084605 6393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:52.086763 6393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:52.086931 6393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:52.087184 6393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:52.087523 6393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:52.087559 6393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.087620 6393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:52.087646 6393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.093459 6393 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:38485
I20260807 22:35:52.093534 6707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:38485 every 8 connection(s)
I20260807 22:35:52.093904 6393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:52.095402 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 6393
I20260807 22:35:52.095618 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 5619
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.253@KRBTEST.COM
I20260807 22:35:52.106539 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.094797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:44563 (local address 127.28.82.253:35269)
0807 22:35:52.094845 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.094847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.095477 (+ 630us) server_negotiation.cc:407] Connection header received
0807 22:35:52.097053 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.097054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.097076 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.097104 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.097974 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.098638 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.100085 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.100350 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.100387 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.103185 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.103187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.103188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.103195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.104972 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.105440 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.105441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.105442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.105496 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.105788 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.105789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.105790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.105937 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.105981 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.106337 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.106418 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 22:35:52.107098 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.094663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.28.82.193:44563)
0807 22:35:52.095326 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.095374 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.096938 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.097273 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.097277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.097627 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.097868 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.097873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.098738 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.098739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.099978 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.099981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.100469 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.100470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.100605 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.101290 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.101301 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.103085 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.105064 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.105072 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.105074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.105356 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.105560 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.105561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.105561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.105708 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.106029 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.106036 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.106189 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.106687 (+ 498us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.106871 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":588,"threads_started":1}
I20260807 22:35:52.107827 6712 heartbeater.cc:347] Connected to a master server at 127.28.82.253:35269
I20260807 22:35:52.107939 6712 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.108433 6712 heartbeater.cc:511] Master 127.28.82.253:35269 requested a full tablet report, sending...
I20260807 22:35:52.109004 6190 ts_manager.cc:194] Registered new tserver with Master: 8b06e445ec1c4b1496ae351e79d8f793 (127.28.82.193:38485)
I20260807 22:35:52.109061 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:37031
--local_ip_for_outbound_sockets=127.28.82.194
--tserver_master_addrs=127.28.82.254:36739
--webserver_port=46705
--webserver_interface=127.28.82.194
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab
--principal=oryx/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269,127.28.82.252:44363 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.252@KRBTEST.COM
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:52.113677 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.094663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.28.82.193:43715)
0807 22:35:52.095344 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.095374 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.103609 (+ 8235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.103885 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.103886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.104381 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.104464 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.104466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.105697 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.105698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.106585 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.106588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.107761 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.107762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.107836 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.108230 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.108253 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.109902 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.112454 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.112455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.112456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.112713 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.112915 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.112916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.112916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.112993 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.113266 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.113267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.113315 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.113385 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.113501 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":3,"thread_start_us":269,"threads_started":1}
I20260807 22:35:52.113746 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.094772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:55715 (local address 127.28.82.254:36739)
0807 22:35:52.094826 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.094829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.095679 (+ 850us) server_negotiation.cc:407] Connection header received
0807 22:35:52.103902 (+ 8223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.103904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.103918 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.103949 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.104885 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.105694 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.106962 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.107312 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.107356 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.110508 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.110509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.110509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.110518 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.112319 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.112869 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.112870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.112871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.112938 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.113186 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.113187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.113188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.113326 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.113413 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.113575 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.113649 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 22:35:52.113799 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.094663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:55715)
0807 22:35:52.095611 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.095619 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.103784 (+ 8165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.104016 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.104017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.104716 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.104792 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.104794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.105815 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.105816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.106864 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.106867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.107441 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.107442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.107504 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.108908 (+ 1404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.108914 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.110388 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.112406 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.112407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.112407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.112763 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.113008 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.113009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.113009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.113076 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.113408 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.113409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.113453 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.113543 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.113648 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":33,"thread_start_us":503,"threads_started":1}
I20260807 22:35:52.114075 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.094779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:43715 (local address 127.28.82.252:44363)
0807 22:35:52.095021 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.095022 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.095502 (+ 480us) server_negotiation.cc:407] Connection header received
0807 22:35:52.103743 (+ 8241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.103745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.103785 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.104061 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.104560 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.105606 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.106895 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.107663 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.107760 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.109996 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.109998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.109998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.110006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.112340 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.112792 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.112793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.112794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.112859 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.113057 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.113058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.113058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.113189 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.113260 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.113839 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.113944 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":175,"threads_started":1}
I20260807 22:35:52.114339 6711 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36739
I20260807 22:35:52.114420 6711 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.114513 6711 heartbeater.cc:511] Master 127.28.82.254:36739 requested a full tablet report, sending...
I20260807 22:35:52.114619 6709 heartbeater.cc:347] Connected to a master server at 127.28.82.252:44363
I20260807 22:35:52.114670 6709 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.114727 6709 heartbeater.cc:511] Master 127.28.82.252:44363 requested a full tablet report, sending...
I20260807 22:35:52.114813 5332 ts_manager.cc:194] Re-registered known tserver with Master: 8b06e445ec1c4b1496ae351e79d8f793 (127.28.82.193:38485)
I20260807 22:35:52.115099 6455 ts_manager.cc:194] Registered new tserver with Master: 8b06e445ec1c4b1496ae351e79d8f793 (127.28.82.193:38485)
I20260807 22:35:52.115162 5332 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:55715
W20260807 22:35:52.195602 6726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:52.195837 6726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:52.195873 6726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:52.197008 6726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:52.197088 6726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:52.197135 6726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:52.198645 6726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab
--principal=oryx/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:37031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=46705
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269,127.28.82.252:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:52.198917 6726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:52.199159 6726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:52.204372 6735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:52.204387 6738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:52.204389 6736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:52.204715 6726 server_base.cc:1034] running on GCE node
I20260807 22:35:52.204952 6726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:52.205188 6726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:52.206350 6726 hybrid_clock.cc:648] HybridClock initialized: now 1786142152206336 us; error 24 us; skew 500 ppm
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:52.208998 6726 init.cc:377] Logged in from keytab as oryx/127.28.82.194@KRBTEST.COM (short username oryx)
I20260807 22:35:52.209476 6726 webserver.cc:460] Webserver started at http://127.28.82.194:46705/ using document root <none> and password file <none>
I20260807 22:35:52.209785 6726 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:52.209872 6726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:52.211455 6726 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:52.212224 6746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.212428 6726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:52.212532 6726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "1038c95d4b5445c88e918ea5fcf5ffdc"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "885c24f71047fe9be82a8d759e6cba33"
server_key_iv: "1c857bfae612450add03cabb2dbb27b6"
server_key_version: "encryptionkey@0"
I20260807 22:35:52.212890 6726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:52.225346 6726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:52.227458 6726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:52.227622 6726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:52.227839 6726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:52.228142 6726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:52.228197 6726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.228237 6726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:52.228264 6726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.235090 6726 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:37031
I20260807 22:35:52.235216 6917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:37031 every 8 connection(s)
I20260807 22:35:52.235545 6726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:52.236405 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 6726
I20260807 22:35:52.236590 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 5810
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.253@KRBTEST.COM
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:52.251139 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.236713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:59785 (local address 127.28.82.253:35269)
0807 22:35:52.237277 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.237279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.238910 (+ 1631us) server_negotiation.cc:407] Connection header received
0807 22:35:52.238998 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.238999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.239013 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.239041 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.240241 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.240885 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.242204 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.242495 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.242534 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.245388 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.245389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.245389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.245397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.249393 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.250046 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.250047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.250048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.250114 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.250360 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.250361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.250361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.250518 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.250571 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.250917 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.251000 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538}
I20260807 22:35:52.251888 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.236722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:45109 (local address 127.28.82.254:36739)
0807 22:35:52.236770 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.236772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.237623 (+ 851us) server_negotiation.cc:407] Connection header received
0807 22:35:52.240299 (+ 2676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.240301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.240317 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.240353 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.240854 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.241826 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.243583 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.244006 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.244060 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.247610 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.247612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.247613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.247622 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.250011 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.250552 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.250553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.250554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.250629 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.250915 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.250916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.250917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.251084 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.251152 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.251619 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.251743 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:52.252161 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.236405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.28.82.194:59785)
0807 22:35:52.238858 (+ 2453us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.238868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.238934 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.239205 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.239209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.239742 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.240127 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.240135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.241005 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.241007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.242112 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.242116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.242622 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.242623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.242752 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.243363 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.243373 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.245292 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.249671 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.249679 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.249681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.249972 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.250178 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.250178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.250179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.250298 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.250631 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.250637 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.250789 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.251498 (+ 709us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.251678 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2421,"thread_start_us":110,"threads_started":1}
I20260807 22:35:52.253536 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.236406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:45109)
0807 22:35:52.237439 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.237519 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.238741 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.240438 (+ 1697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.240439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.240664 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.240746 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.240748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.241936 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.241937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.243466 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.243470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.244159 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.244160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.244246 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.244837 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.244843 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.247476 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.250133 (+ 2657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.250134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.250135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.250418 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.250735 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.250735 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.250736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.250816 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.251194 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.251195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.251252 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.251981 (+ 729us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.252091 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"thread_start_us":97,"threads_started":1}
I20260807 22:35:52.254978 6920 heartbeater.cc:347] Connected to a master server at 127.28.82.253:35269
I20260807 22:35:52.255118 6919 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36739
I20260807 22:35:52.255218 6919 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.255405 6919 heartbeater.cc:511] Master 127.28.82.254:36739 requested a full tablet report, sending...
I20260807 22:35:52.255568 6920 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.255630 6920 heartbeater.cc:511] Master 127.28.82.253:35269 requested a full tablet report, sending...
I20260807 22:35:52.255810 5332 ts_manager.cc:194] Re-registered known tserver with Master: 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031)
I20260807 22:35:52.255913 6190 ts_manager.cc:194] Registered new tserver with Master: 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031)
I20260807 22:35:52.256363 5332 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:45109
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.252@KRBTEST.COM
I20260807 22:35:52.258602 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:38963
--local_ip_for_outbound_sockets=127.28.82.195
--tserver_master_addrs=127.28.82.254:36739
--webserver_port=37111
--webserver_interface=127.28.82.195
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab
--principal=oryx/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269,127.28.82.252:44363 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:35:52.260946 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.236746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.28.82.194:55785)
0807 22:35:52.249563 (+ 12817us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.249576 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.249618 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.253331 (+ 3713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.253332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.253521 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.253584 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.253586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.254311 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.254311 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.255102 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.255105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.256085 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.256086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.256151 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.256577 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.256582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.257887 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.259685 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.259686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.259687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.259914 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.260168 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.260168 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.260169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.260276 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.260575 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.260576 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.260626 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.260735 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.260863 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12793,"thread_start_us":72,"threads_started":1}
I20260807 22:35:52.261027 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.237214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:55785 (local address 127.28.82.252:44363)
0807 22:35:52.252110 (+ 14896us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.252112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.252121 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:35:52.252146 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.252147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.252174 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.252273 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.253699 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.254230 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.255185 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.255581 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.255668 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.257973 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.257974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.257975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.257983 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.259586 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.260039 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.260040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.260040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.260094 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.260376 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.260377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.260378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.260497 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.260565 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.260783 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.260903 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14867}
I20260807 22:35:52.261606 6918 heartbeater.cc:347] Connected to a master server at 127.28.82.252:44363
I20260807 22:35:52.261677 6918 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.261742 6918 heartbeater.cc:511] Master 127.28.82.252:44363 requested a full tablet report, sending...
I20260807 22:35:52.262069 6455 ts_manager.cc:194] Registered new tserver with Master: 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031)
W20260807 22:35:52.343519 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:52.343773 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:52.343809 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:52.345034 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:52.345109 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:52.345160 6926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:52.346674 6926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab
--principal=oryx/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:38963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=37111
--tserver_master_addrs=127.28.82.254:36739,127.28.82.253:35269,127.28.82.252:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:52.346918 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:52.347163 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:52.352870 6935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:52.352934 6926 server_base.cc:1034] running on GCE node
W20260807 22:35:52.352834 6937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:52.352885 6934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:52.353293 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:52.353551 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:52.354791 6926 hybrid_clock.cc:648] HybridClock initialized: now 1786142152354770 us; error 50 us; skew 500 ppm
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:52.357658 6926 init.cc:377] Logged in from keytab as oryx/127.28.82.195@KRBTEST.COM (short username oryx)
I20260807 22:35:52.358076 6926 webserver.cc:460] Webserver started at http://127.28.82.195:37111/ using document root <none> and password file <none>
I20260807 22:35:52.358263 6926 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:52.358318 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:52.359879 6926 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:52.361588 6946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.361821 6926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:52.361918 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c"
format_stamp: "Formatted at 2026-08-07 22:35:51 on dist-test-slave-7kzw"
server_key: "b890136b485ed559bc9550bb5008b9a5"
server_key_iv: "13bf81c93da134449aafa2d44fd35525"
server_key_version: "encryptionkey@0"
I20260807 22:35:52.362282 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:52.371305 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:52.373795 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:52.373917 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:52.374127 6926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:35:52.374559 6926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:52.374626 6926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.374662 6926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:52.374680 6926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:52.378275 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.374658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.28.82.252:44363)
0807 22:35:52.374714 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.374716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.374735 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:35:52.374766 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.374768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.374795 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.374858 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:52.375051 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.375577 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.377068 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.377813 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.377915 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.378008 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.378113 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.378172 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 22:35:52.378275 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.374572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.0.0.1:55918)
0807 22:35:52.374635 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.374648 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.374703 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.374893 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.374894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.374897 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:52.374948 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.374952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.375681 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.375683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.376946 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.376951 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.377929 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.377931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.378019 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.378024 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.378094 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.378175 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 22:35:52.378763 6467 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 22:35:52.378952 6507 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21af200769d44c058aa2fb4efd2fd9a2. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:52.379084 6507 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: LEARNER
I20260807 22:35:52.383147 6926 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:38963
I20260807 22:35:52.383219 7115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:38963 every 8 connection(s)
I20260807 22:35:52.383584 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:52.385413 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 6926
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.253@KRBTEST.COM
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
Aug 07 22:35:52 dist-test-slave-7kzw krb5kdc[5255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142152, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.252@KRBTEST.COM
I20260807 22:35:52.397336 6259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.384772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:41113 (local address 127.28.82.253:35269)
0807 22:35:52.384843 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.384845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.384855 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:52.385579 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.385582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.385602 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.385681 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.386776 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.387286 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.388455 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.388759 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.388806 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.393395 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.393396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.393397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.393406 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.395305 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.395860 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.395862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.395863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.395930 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.396321 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.396323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.396323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.396466 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.396515 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.396984 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.397072 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:35:52.397421 7123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.384466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:52593)
0807 22:35:52.384977 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.384985 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.385524 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.386394 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.386395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.386594 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.386667 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.386673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.387885 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.387886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.388762 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.388764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.389268 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.389269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.389346 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.390258 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.390264 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.392847 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.395703 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.395704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.395705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.396203 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.396426 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.396427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.396428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.396482 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.396789 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.396790 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.396839 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.396923 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.397148 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":100,"thread_start_us":131,"threads_started":1}
I20260807 22:35:52.398268 7120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.384466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.28.82.195:41113)
0807 22:35:52.384685 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.384737 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.385379 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.385801 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.385806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.386347 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.386666 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.386672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.387401 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.387402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.388349 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.388352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.388900 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.388901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.389055 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.389993 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.390002 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.393314 (+ 3312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.395386 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.395394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.395397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.395756 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.396008 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.396009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.396009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.396229 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.396570 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.396578 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.396795 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.397998 (+ 1203us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.398122 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260807 22:35:52.398439 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.384760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:52593 (local address 127.28.82.254:36739)
0807 22:35:52.384813 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.384816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.385035 (+ 219us) server_negotiation.cc:407] Connection header received
0807 22:35:52.385588 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.385590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.385604 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.385655 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.386792 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.387792 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.388856 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.389140 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.389209 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.392983 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.392986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.392987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.392996 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.395611 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.396287 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.396288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.396288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.396361 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.396551 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.396562 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.396562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.396694 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.396759 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.398258 (+ 1499us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.398342 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:52.398991 7117 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36739
I20260807 22:35:52.399096 7117 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.399219 7118 heartbeater.cc:347] Connected to a master server at 127.28.82.253:35269
I20260807 22:35:52.399268 7118 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.399315 7117 heartbeater.cc:511] Master 127.28.82.254:36739 requested a full tablet report, sending...
I20260807 22:35:52.399343 7118 heartbeater.cc:511] Master 127.28.82.253:35269 requested a full tablet report, sending...
I20260807 22:35:52.399735 5332 ts_manager.cc:194] Re-registered known tserver with Master: e0d66649e0dd4ce1bd1eea96ddf2c74c (127.28.82.195:38963)
I20260807 22:35:52.399761 6190 ts_manager.cc:194] Registered new tserver with Master: e0d66649e0dd4ce1bd1eea96ddf2c74c (127.28.82.195:38963)
I20260807 22:35:52.400224 5332 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:52593
I20260807 22:35:52.401503 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.388551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.28.82.253:35269)
0807 22:35:52.392092 (+ 3541us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.392094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.392261 (+ 167us) server_negotiation.cc:407] Connection header received
0807 22:35:52.392286 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.392286 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.392300 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.392354 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.393084 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.393839 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.395252 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.395565 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.395631 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.397358 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.397359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.397360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.397373 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.399492 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.400386 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.400387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.400388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.400450 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.400691 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.400692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.400692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.400850 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.400950 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.401308 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.401405 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3518,"thread_start_us":118,"threads_started":1}
I20260807 22:35:52.402346 7132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.388880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55920 (local address 127.28.82.252:44363)
0807 22:35:52.390237 (+ 1357us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.390240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.390250 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:52.390271 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.390272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.390285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.390357 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.391299 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.391938 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.393946 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.394286 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.394357 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.397491 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.397492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.397493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.397501 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.400722 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.401157 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.401158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.401158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.401226 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.401432 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.401433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.401434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.401585 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.401949 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.402143 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.402234 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1336,"thread_start_us":108,"threads_started":1}
I20260807 22:35:52.404996 7121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.384467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.28.82.195:58341)
0807 22:35:52.384730 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.384739 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.385379 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.385866 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.385868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.386385 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:52.386667 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.386673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.387846 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.387847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.388862 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.388865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.390391 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.390392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.390495 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.390986 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:52.390992 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:52.401320 (+ 10328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:52.403502 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.403503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.403504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.403745 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.404006 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:52.404007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:52.404008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:52.404154 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:52.404541 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:52.404542 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:52.404595 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.404744 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.404861 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":69,"thread_start_us":81,"threads_started":1}
I20260807 22:35:52.405016 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.384848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:58341 (local address 127.28.82.252:44363)
0807 22:35:52.384896 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.384898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.384906 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:52.385664 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.385665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.385683 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.385733 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.386791 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.387734 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.388975 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.390245 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.390295 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.401432 (+ 11137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.401434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.401434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.401443 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.403382 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.403843 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.403844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.403844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.403906 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.404293 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.404294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.404294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.404449 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.404515 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.404808 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.404913 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:35:52.405349 7116 heartbeater.cc:347] Connected to a master server at 127.28.82.252:44363
I20260807 22:35:52.405409 7116 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:52.405491 7116 heartbeater.cc:511] Master 127.28.82.252:44363 requested a full tablet report, sending...
I20260807 22:35:52.405864 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.389198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.28.82.254:36739)
0807 22:35:52.389468 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.389470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.394589 (+ 5119us) server_negotiation.cc:407] Connection header received
0807 22:35:52.394616 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.394617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.394634 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.394700 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.395703 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.396492 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.398882 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.399188 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.399253 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.402007 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.402008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.402009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.402017 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.404341 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.404782 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.404783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.404784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.404855 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.405135 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.405137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.405137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.405294 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.405391 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.405687 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.405786 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":180,"threads_started":1}
I20260807 22:35:52.405874 6455 ts_manager.cc:194] Registered new tserver with Master: e0d66649e0dd4ce1bd1eea96ddf2c74c (127.28.82.195:38963)
I20260807 22:35:52.407723 5332 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:52.409420 5332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:52.410560 6130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: NON_VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: true } } }
I20260807 22:35:52.410705 6130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:52.412199 6129 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: attempting to promote NON_VOTER f6b627b3aca44300b46e08e129409427 to VOTER
I20260807 22:35:52.412464 6129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } }
I20260807 22:35:52.413041 6467 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 LEARNER]: Refusing update from remote peer 21af200769d44c058aa2fb4efd2fd9a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 22:35:52.413291 6128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 22:35:52.413421 6203 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Refusing update from remote peer 21af200769d44c058aa2fb4efd2fd9a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 22:35:52.413623 6130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 22:35:52.414039 6128 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f6b627b3aca44300b46e08e129409427 (127.28.82.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:52.414237 6467 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f6b627b3aca44300b46e08e129409427 (127.28.82.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:52.414803 6203 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f6b627b3aca44300b46e08e129409427 (127.28.82.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:52.415313 6128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:52.415410 6128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:52.415477 6128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:52.415436 6507 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:52.415545 6128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:52.415546 6507 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:35:52.415599 6128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:52.415654 6128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:52.415817 6244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21af200769d44c058aa2fb4efd2fd9a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:52.415905 6244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:35:52.421372 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.417011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.28.82.193:38485)
0807 22:35:52.417412 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.417418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.417430 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:35:52.417477 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.417479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.417501 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.417611 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:52.417767 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.418902 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.420260 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.420989 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.421053 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.421141 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.421240 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.421287 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
I20260807 22:35:52.421368 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.416862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:38485 (local address 127.0.0.1:43794)
0807 22:35:52.416908 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.416921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.416972 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.417612 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.417613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.417615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:52.417683 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.417685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.418995 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.418997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.420162 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.420166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.421070 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.421071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.421144 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.421148 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.421219 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.421278 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":1}
I20260807 22:35:52.421706 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.416987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38963 (local address 127.0.0.1:38680)
0807 22:35:52.417201 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.417212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.417261 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.417961 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.417962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.417965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:52.418004 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.418005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.419266 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.419267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.420358 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.420361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.421445 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.421446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.421548 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.421552 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.421601 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.421641 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":148,"threads_started":1}
I20260807 22:35:52.421898 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.417367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.28.82.194:37031)
0807 22:35:52.417553 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.417557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.417568 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:35:52.417610 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.417611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.417628 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.417710 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:52.418229 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.419114 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.420106 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.420975 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.421053 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.421137 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.421413 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.421684 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260807 22:35:52.422142 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.417073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38680 (local address 127.28.82.195:38963)
0807 22:35:52.417779 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.417783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.417794 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:35:52.417844 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.417846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.417863 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.417951 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:52.418097 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.419191 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.420455 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.421076 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.421152 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.421972 (+ 820us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.422005 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.422058 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":127,"threads_started":1}
I20260807 22:35:52.422263 6614 tablet_service.cc:1467] Processing CreateTablet for tablet da62676fbe7e4359b90bf9f5e7f2ca63 (DEFAULT_TABLE table=test-table [id=98775023ade34631b755a2aca970e6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:52.422693 6614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da62676fbe7e4359b90bf9f5e7f2ca63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:52.425606 7154 tablet_bootstrap.cc:492] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: Bootstrap starting.
I20260807 22:35:52.426105 7152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.417286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:37031 (local address 127.0.0.1:34422)
0807 22:35:52.417415 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.417426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.417475 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.418065 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.418066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.418069 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:52.418133 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.418134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.419222 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.419223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.420001 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.420004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.421176 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.421177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.421242 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.421247 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.421297 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.421336 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":39,"thread_start_us":79,"threads_started":1}
I20260807 22:35:52.426309 7154 tablet_bootstrap.cc:654] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:52.426533 7013 tablet_service.cc:1467] Processing CreateTablet for tablet da62676fbe7e4359b90bf9f5e7f2ca63 (DEFAULT_TABLE table=test-table [id=98775023ade34631b755a2aca970e6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:52.426687 7154 log.cc:824] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:52.426908 7013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da62676fbe7e4359b90bf9f5e7f2ca63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:52.426837 6816 tablet_service.cc:1467] Processing CreateTablet for tablet da62676fbe7e4359b90bf9f5e7f2ca63 (DEFAULT_TABLE table=test-table [id=98775023ade34631b755a2aca970e6f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:52.427205 6816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da62676fbe7e4359b90bf9f5e7f2ca63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:52.427672 7154 tablet_bootstrap.cc:492] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: No bootstrap required, opened a new log
I20260807 22:35:52.427748 7154 ts_tablet_manager.cc:1397] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:52.430380 7157 tablet_bootstrap.cc:492] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Bootstrap starting.
I20260807 22:35:52.430356 7154 raft_consensus.cc:348] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.430572 7154 raft_consensus.cc:374] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:52.430621 7154 raft_consensus.cc:729] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b06e445ec1c4b1496ae351e79d8f793, State: Initialized, Role: FOLLOWER
I20260807 22:35:52.430742 7154 consensus_queue.cc:260] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.431038 7154 ts_tablet_manager.cc:1428] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 22:35:52.431316 6711 heartbeater.cc:503] Master 127.28.82.254:36739 was elected leader, sending a full tablet report...
I20260807 22:35:52.431388 7157 tablet_bootstrap.cc:654] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:52.431751 7157 log.cc:824] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:52.432685 7158 tablet_bootstrap.cc:492] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Bootstrap starting.
I20260807 22:35:52.432945 7157 tablet_bootstrap.cc:492] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: No bootstrap required, opened a new log
I20260807 22:35:52.433043 7157 ts_tablet_manager.cc:1397] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 22:35:52.433243 7158 tablet_bootstrap.cc:654] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:52.433558 7158 log.cc:824] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:52.434265 7158 tablet_bootstrap.cc:492] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: No bootstrap required, opened a new log
I20260807 22:35:52.434320 7158 ts_tablet_manager.cc:1397] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:52.435333 7157 raft_consensus.cc:348] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.435499 7157 raft_consensus.cc:374] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:52.435577 7157 raft_consensus.cc:729] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1038c95d4b5445c88e918ea5fcf5ffdc, State: Initialized, Role: FOLLOWER
I20260807 22:35:52.435717 7157 consensus_queue.cc:260] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.435858 7158 raft_consensus.cc:348] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.435983 7158 raft_consensus.cc:374] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:52.436012 7158 raft_consensus.cc:729] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d66649e0dd4ce1bd1eea96ddf2c74c, State: Initialized, Role: FOLLOWER
I20260807 22:35:52.436048 7157 ts_tablet_manager.cc:1428] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 22:35:52.436126 7158 consensus_queue.cc:260] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.436208 6919 heartbeater.cc:503] Master 127.28.82.254:36739 was elected leader, sending a full tablet report...
I20260807 22:35:52.436483 7158 ts_tablet_manager.cc:1428] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:35:52.436623 7117 heartbeater.cc:503] Master 127.28.82.254:36739 was elected leader, sending a full tablet report...
I20260807 22:35:52.485649 7159 raft_consensus.cc:482] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:52.485826 7159 raft_consensus.cc:504] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.486703 7159 leader_election.cc:288] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0d66649e0dd4ce1bd1eea96ddf2c74c (127.28.82.195:38963), 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031)
I20260807 22:35:52.489578 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.486996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:41701 (local address 127.28.82.195:38963)
0807 22:35:52.487059 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.487061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.487074 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:35:52.487110 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.487111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.487123 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.487190 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:52.487397 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.487885 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.488853 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.489220 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.489308 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.489317 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.489469 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.489506 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:35:52.489578 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.486898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38963 (local address 127.28.82.193:41701)
0807 22:35:52.486966 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.486979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.487029 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.487214 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.487215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.487218 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:52.487313 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.487315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.487976 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.487977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.488748 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.488750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.489305 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.489306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.489388 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.489392 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.489462 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.489505 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:35:52.489784 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.487049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:37031 (local address 127.28.82.193:48987)
0807 22:35:52.487098 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:52.487110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:52.487177 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:52.487405 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:52.487406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:52.487408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:52.487477 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.487479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.488174 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.488175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.488879 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:52.488881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.489497 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.489499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:52.489588 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.489593 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:35:52.489663 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:35:52.489704 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 22:35:52.489781 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.487213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:48987 (local address 127.28.82.194:37031)
0807 22:35:52.487285 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.487289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.487302 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:35:52.487323 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.487324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.487331 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.487400 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:52.487559 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.488068 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.488989 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.489381 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.489475 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.489484 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.489682 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.489720 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:35:52.490170 6846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da62676fbe7e4359b90bf9f5e7f2ca63" candidate_uuid: "8b06e445ec1c4b1496ae351e79d8f793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" is_pre_election: true
I20260807 22:35:52.490360 6846 raft_consensus.cc:2454] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b06e445ec1c4b1496ae351e79d8f793 in term 0.
I20260807 22:35:52.490433 7041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da62676fbe7e4359b90bf9f5e7f2ca63" candidate_uuid: "8b06e445ec1c4b1496ae351e79d8f793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" is_pre_election: true
I20260807 22:35:52.490633 7041 raft_consensus.cc:2454] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b06e445ec1c4b1496ae351e79d8f793 in term 0.
I20260807 22:35:52.490679 6553 leader_election.cc:302] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1038c95d4b5445c88e918ea5fcf5ffdc, 8b06e445ec1c4b1496ae351e79d8f793; no voters:
I20260807 22:35:52.490829 7159 raft_consensus.cc:2781] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:52.490898 7159 raft_consensus.cc:482] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:52.490916 7159 raft_consensus.cc:3037] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:52.491690 7159 raft_consensus.cc:504] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.491814 7159 leader_election.cc:288] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [CANDIDATE]: Term 1 election: Requested vote from peers e0d66649e0dd4ce1bd1eea96ddf2c74c (127.28.82.195:38963), 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031)
I20260807 22:35:52.492121 7041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da62676fbe7e4359b90bf9f5e7f2ca63" candidate_uuid: "8b06e445ec1c4b1496ae351e79d8f793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c"
I20260807 22:35:52.492209 6846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da62676fbe7e4359b90bf9f5e7f2ca63" candidate_uuid: "8b06e445ec1c4b1496ae351e79d8f793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc"
I20260807 22:35:52.492228 7041 raft_consensus.cc:3037] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:52.492308 6846 raft_consensus.cc:3037] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:52.493183 7041 raft_consensus.cc:2454] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b06e445ec1c4b1496ae351e79d8f793 in term 1.
I20260807 22:35:52.493323 6846 raft_consensus.cc:2454] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b06e445ec1c4b1496ae351e79d8f793 in term 1.
I20260807 22:35:52.493335 6553 leader_election.cc:302] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b06e445ec1c4b1496ae351e79d8f793, e0d66649e0dd4ce1bd1eea96ddf2c74c; no voters:
I20260807 22:35:52.493435 7159 raft_consensus.cc:2781] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:52.493769 7159 raft_consensus.cc:686] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 1 LEADER]: Becoming Leader. State: Replica: 8b06e445ec1c4b1496ae351e79d8f793, State: Running, Role: LEADER
I20260807 22:35:52.493862 7159 consensus_queue.cc:237] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } }
I20260807 22:35:52.494937 5332 catalog_manager.cc:5495] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b06e445ec1c4b1496ae351e79d8f793 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "8b06e445ec1c4b1496ae351e79d8f793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } health_report { overall_health: UNKNOWN } } }
W20260807 22:35:52.501714 6921 tablet.cc:2369] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:52.516714 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.513834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.28.82.193:38485)
0807 22:35:52.513911 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.513913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.513923 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:52.513972 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.513972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.513992 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.514071 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:52.514207 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.514831 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.515694 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.515889 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.515956 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.516304 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:52.516414 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:52.516483 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.516568 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.516587 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:35:52.518678 6846 raft_consensus.cc:1261] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 1 FOLLOWER]: Refusing update from remote peer 8b06e445ec1c4b1496ae351e79d8f793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:52.518704 7041 raft_consensus.cc:1261] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 FOLLOWER]: Refusing update from remote peer 8b06e445ec1c4b1496ae351e79d8f793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:52.519069 7164 consensus_queue.cc:1035] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:52.519229 7159 consensus_queue.cc:1035] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:52.521855 7167 mvcc.cc:205] Tried to move back new op lower bound from 7316038256713367552 to 7316038256615673856. Current Snapshot: MvccSnapshot[applied={T|T < 7316038256713367552}]
I20260807 22:35:52.522815 7168 mvcc.cc:205] Tried to move back new op lower bound from 7316038256713367552 to 7316038256615673856. Current Snapshot: MvccSnapshot[applied={T|T < 7316038256713367552}]
I20260807 22:35:52.524821 5329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:52.524937 5329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:52.526600 5329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21af200769d44c058aa2fb4efd2fd9a2: Sending DeleteTablet for 3 replicas of tablet da62676fbe7e4359b90bf9f5e7f2ca63
I20260807 22:35:52.526944 6816 tablet_service.cc:1514] Processing DeleteTablet for tablet da62676fbe7e4359b90bf9f5e7f2ca63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:52 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:34422
I20260807 22:35:52.526958 6614 tablet_service.cc:1514] Processing DeleteTablet for tablet da62676fbe7e4359b90bf9f5e7f2ca63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:52 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:43794
I20260807 22:35:52.527127 6612 tablet_service.cc:1514] Processing DeleteTablet for tablet da62676fbe7e4359b90bf9f5e7f2ca63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:52 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:43794
I20260807 22:35:52.527253 7179 tablet_replica.cc:330] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: stopping tablet replica
I20260807 22:35:52.527237 7013 tablet_service.cc:1514] Processing DeleteTablet for tablet da62676fbe7e4359b90bf9f5e7f2ca63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:52 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:38680
I20260807 22:35:52.527367 7179 raft_consensus.cc:2229] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:52.527460 7179 raft_consensus.cc:2258] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:52.527669 7180 tablet_replica.cc:330] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: stopping tablet replica
I20260807 22:35:52.528257 7179 ts_tablet_manager.cc:1905] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:52.528641 7182 tablet_replica.cc:330] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: stopping tablet replica
I20260807 22:35:52.528667 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 5274
I20260807 22:35:52.528820 7182 raft_consensus.cc:2229] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:52.530092 7180 raft_consensus.cc:2229] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:52.530210 7180 raft_consensus.cc:2258] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:52.530953 7179 ts_tablet_manager.cc:1918] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:52.531016 7179 log.cc:1196] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/da62676fbe7e4359b90bf9f5e7f2ca63
I20260807 22:35:52.531250 7179 ts_tablet_manager.cc:1939] T da62676fbe7e4359b90bf9f5e7f2ca63 P 8b06e445ec1c4b1496ae351e79d8f793: Deleting consensus metadata
I20260807 22:35:52.532491 7180 ts_tablet_manager.cc:1905] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:52.534719 7180 ts_tablet_manager.cc:1918] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:52.534785 7180 log.cc:1196] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/da62676fbe7e4359b90bf9f5e7f2ca63
I20260807 22:35:52.535022 7180 ts_tablet_manager.cc:1939] T da62676fbe7e4359b90bf9f5e7f2ca63 P 1038c95d4b5445c88e918ea5fcf5ffdc: Deleting consensus metadata
I20260807 22:35:52.538985 7182 raft_consensus.cc:2258] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:52.543110 7182 ts_tablet_manager.cc:1905] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:52.544952 7182 ts_tablet_manager.cc:1918] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:52.545120 7182 log.cc:1196] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/da62676fbe7e4359b90bf9f5e7f2ca63
I20260807 22:35:52.545555 7182 ts_tablet_manager.cc:1939] T da62676fbe7e4359b90bf9f5e7f2ca63 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Deleting consensus metadata
W20260807 22:35:52.548974 6951 connection.cc:535] server connection from 127.0.0.1:38680 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 22:35:52.548990 6749 connection.cc:535] server connection from 127.0.0.1:34422 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:35:52.549372 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.527866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.28.82.252:44363)
0807 22:35:52.527915 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.527916 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.527924 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:52.527943 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.527944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.527955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.528004 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.541936 (+ 13932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.542763 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.543960 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.544257 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.544311 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.546164 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.546165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.546166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.546174 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.547784 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.548162 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.548163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.548164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.548324 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.548545 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.548546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.548547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.548708 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.548816 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.549188 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.549280 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 22:35:52.549376 6552 connection.cc:535] server connection from 127.0.0.1:43794 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:35:52.560916 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.552853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.28.82.252:44363)
0807 22:35:52.552888 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.552890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.552946 (+ 56us) server_negotiation.cc:407] Connection header received
0807 22:35:52.553035 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.553036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.553046 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.553101 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.553848 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.554699 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.556029 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.556344 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.556407 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.557983 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.557985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.557985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.557993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.559614 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.560013 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.560014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.560015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.560084 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.560318 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.560320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.560320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.560460 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.560550 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.560730 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.560808 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 22:35:52.561520 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:52.552731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50104 (local address 127.28.82.253:35269)
0807 22:35:52.552784 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:52.552786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:52.553002 (+ 216us) server_negotiation.cc:407] Connection header received
0807 22:35:52.553097 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:52.553098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:52.553120 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:52.553214 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:52.554646 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.555439 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.556372 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:52.556667 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:52.556751 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:52.558236 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:52.558237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:52.558238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:52.558246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:52.559876 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.560261 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.560262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.560262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.560320 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:52.560496 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:52.560497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:52.560497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:52.560636 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:52.560752 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:52.561332 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 22:35:52.561425 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 22:35:53.524685 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:53.524304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:41259)
0807 22:35:53.524458 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:53.524547 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260807 22:35:53.524724 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:53.524470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:59095)
0807 22:35:53.524585 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:53.524659 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
W20260807 22:35:53.524919 7117 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:36739 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
W20260807 22:35:53.525036 6919 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:36739 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:53.527256 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:53.527041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:46851)
0807 22:35:53.527149 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:53.527218 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":51,"threads_started":1}
W20260807 22:35:53.527454 6711 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:36739 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:54.027151 7204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 21af200769d44c058aa2fb4efd2fd9a2)
I20260807 22:35:54.027318 7204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } }
I20260807 22:35:54.027437 7205 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 21af200769d44c058aa2fb4efd2fd9a2)
I20260807 22:35:54.027531 7205 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } }
I20260807 22:35:54.027724 7204 leader_election.cc:288] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739), f6b627b3aca44300b46e08e129409427 (127.28.82.252:44363)
I20260807 22:35:54.027755 7205 leader_election.cc:288] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739), 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253:35269)
I20260807 22:35:54.028169 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.027875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58384)
0807 22:35:54.028049 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.028117 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":56,"threads_started":1}
I20260807 22:35:54.028194 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.027936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58394)
0807 22:35:54.028049 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.028144 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":48,"threads_started":1}
W20260807 22:35:54.028416 6438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
W20260807 22:35:54.028730 6171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:54.028936 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.028827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58396)
0807 22:35:54.028882 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.028908 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 22:35:54.029006 6438 leader_election.cc:334] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739): Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:54.029389 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.029274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58400)
0807 22:35:54.029332 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.029363 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 22:35:54.029522 6171 leader_election.cc:334] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739): Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:54.031654 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.027941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35269 (local address 127.0.0.1:47330)
0807 22:35:54.028176 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.028188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:54.028283 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:54.028536 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:54.028537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:54.028540 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:54.028604 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:54.028607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:54.029550 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:54.029551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:54.030829 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:54.030832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:54.031405 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:54.031407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:54.031464 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:54.031466 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:35:54.031522 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 22:35:54.031580 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":39,"threads_started":1}
I20260807 22:35:54.031661 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.028035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.28.82.253:35269)
0807 22:35:54.028117 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:54.028119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:54.028250 (+ 131us) server_negotiation.cc:407] Connection header received
0807 22:35:54.028380 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:54.028381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:54.028393 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:54.028478 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:54.028698 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:54.029455 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:54.030939 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:54.031314 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:54.031382 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:54.031425 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:54.031536 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 22:35:54.031584 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64,"thread_start_us":37,"threads_started":1}
I20260807 22:35:54.031767 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.027892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:44363 (local address 127.0.0.1:55078)
0807 22:35:54.028309 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.028320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:54.028377 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:54.028591 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:54.028592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:54.028595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:54.028649 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:54.028651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:54.029670 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:54.029671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:54.030829 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:54.030832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:54.031443 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:54.031444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:54.031514 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:54.031518 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:54.031578 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 22:35:54.031608 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":49,"threads_started":1}
I20260807 22:35:54.031780 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.028113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.28.82.252:44363)
0807 22:35:54.028254 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:54.028256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:54.028379 (+ 123us) server_negotiation.cc:407] Connection header received
0807 22:35:54.028435 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:54.028436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:54.028446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:54.028548 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:54.028753 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:54.029591 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:54.030947 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:54.031326 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:54.031422 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:54.031431 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:54.031594 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 22:35:54.031629 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":42,"threads_started":1}
I20260807 22:35:54.032106 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6b627b3aca44300b46e08e129409427" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "20c8d9158ef54b8daf236ea65cafaf55" is_pre_election: true
I20260807 22:35:54.032222 6203 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b627b3aca44300b46e08e129409427 in term 1.
I20260807 22:35:54.032162 6467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "20c8d9158ef54b8daf236ea65cafaf55" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f6b627b3aca44300b46e08e129409427" is_pre_election: true
I20260807 22:35:54.032284 6467 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c8d9158ef54b8daf236ea65cafaf55 in term 1.
I20260807 22:35:54.032375 6435 leader_election.cc:302] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20c8d9158ef54b8daf236ea65cafaf55, f6b627b3aca44300b46e08e129409427; no voters: 21af200769d44c058aa2fb4efd2fd9a2
I20260807 22:35:54.032466 7205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 22:35:54.032464 6168 leader_election.cc:302] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20c8d9158ef54b8daf236ea65cafaf55, f6b627b3aca44300b46e08e129409427; no voters: 21af200769d44c058aa2fb4efd2fd9a2
I20260807 22:35:54.032506 7205 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 21af200769d44c058aa2fb4efd2fd9a2)
I20260807 22:35:54.032528 7205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:35:54.032589 7204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 22:35:54.032626 7204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 21af200769d44c058aa2fb4efd2fd9a2)
I20260807 22:35:54.032656 7204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:35:54.033286 7205 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } }
I20260807 22:35:54.033376 7204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } }
I20260807 22:35:54.033437 7205 leader_election.cc:288] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 2 election: Requested vote from peers 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739), 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253:35269)
I20260807 22:35:54.033535 7204 leader_election.cc:288] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [CANDIDATE]: Term 2 election: Requested vote from peers 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739), f6b627b3aca44300b46e08e129409427 (127.28.82.252:44363)
I20260807 22:35:54.033684 6467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "20c8d9158ef54b8daf236ea65cafaf55" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f6b627b3aca44300b46e08e129409427"
I20260807 22:35:54.033717 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6b627b3aca44300b46e08e129409427" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "20c8d9158ef54b8daf236ea65cafaf55"
I20260807 22:35:54.033771 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.033635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58422)
0807 22:35:54.033679 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.033713 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 22:35:54.033783 6467 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 20c8d9158ef54b8daf236ea65cafaf55 in current term 2: Already voted for candidate f6b627b3aca44300b46e08e129409427 in this term.
I20260807 22:35:54.033815 6203 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f6b627b3aca44300b46e08e129409427 in current term 2: Already voted for candidate 20c8d9158ef54b8daf236ea65cafaf55 in this term.
I20260807 22:35:54.034065 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.033517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58408)
0807 22:35:54.033977 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.034039 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442}
I20260807 22:35:54.034089 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.033983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58432)
0807 22:35:54.034035 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.034064 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 22:35:54.034189 6171 leader_election.cc:334] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739): Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:54.034256 6171 leader_election.cc:302] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20c8d9158ef54b8daf236ea65cafaf55; no voters: 21af200769d44c058aa2fb4efd2fd9a2, f6b627b3aca44300b46e08e129409427
I20260807 22:35:54.034368 7204 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 22:35:54.034421 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.034314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58448)
0807 22:35:54.034368 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.034399 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 22:35:54.034510 6438 leader_election.cc:334] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739): Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:54.034554 6438 leader_election.cc:302] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6b627b3aca44300b46e08e129409427; no voters: 20c8d9158ef54b8daf236ea65cafaf55, 21af200769d44c058aa2fb4efd2fd9a2
I20260807 22:35:54.034641 7205 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 22:35:54.525517 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.525259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:33933)
0807 22:35:54.525400 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.525446 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260807 22:35:54.525825 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.525519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:43891)
0807 22:35:54.525683 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.525770 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260807 22:35:54.527916 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:54.527747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:52945)
0807 22:35:54.527852 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:54.527876 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260807 22:35:55.526413 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.526207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:49421)
0807 22:35:55.526319 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.526366 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":49,"threads_started":1}
I20260807 22:35:55.526418 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.526154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:40581)
0807 22:35:55.526322 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.526372 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260807 22:35:55.528477 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.528319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:53603)
0807 22:35:55.528416 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.528441 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":47,"threads_started":1}
I20260807 22:35:55.893754 7230 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:55.893854 7230 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } }
I20260807 22:35:55.894065 7230 leader_election.cc:288] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739), 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253:35269)
I20260807 22:35:55.894335 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6b627b3aca44300b46e08e129409427" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "20c8d9158ef54b8daf236ea65cafaf55" is_pre_election: true
I20260807 22:35:55.894456 6203 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b627b3aca44300b46e08e129409427 in term 2.
I20260807 22:35:55.894454 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.894236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58666)
0807 22:35:55.894368 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.894409 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260807 22:35:55.894726 6435 leader_election.cc:302] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c8d9158ef54b8daf236ea65cafaf55, f6b627b3aca44300b46e08e129409427; no voters:
I20260807 22:35:55.894832 7230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 22:35:55.894876 7230 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:55.894907 7230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:35:55.895236 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.895126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58668)
0807 22:35:55.895185 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.895216 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 22:35:55.895347 6438 leader_election.cc:334] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739): Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:55.895900 7230 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } }
I20260807 22:35:55.896050 7230 leader_election.cc:288] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 3 election: Requested vote from peers 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739), 20c8d9158ef54b8daf236ea65cafaf55 (127.28.82.253:35269)
I20260807 22:35:55.896232 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.896134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58678)
0807 22:35:55.896174 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.896207 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 22:35:55.896251 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6b627b3aca44300b46e08e129409427" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "20c8d9158ef54b8daf236ea65cafaf55"
I20260807 22:35:55.896366 6203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:35:55.897019 6203 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b627b3aca44300b46e08e129409427 in term 3.
I20260807 22:35:55.897257 6435 leader_election.cc:302] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c8d9158ef54b8daf236ea65cafaf55, f6b627b3aca44300b46e08e129409427; no voters:
I20260807 22:35:55.897446 7230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:35:55.897526 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.897413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58688)
0807 22:35:55.897471 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.897497 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:35:55.897608 7230 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [term 3 LEADER]: Becoming Leader. State: Replica: f6b627b3aca44300b46e08e129409427, State: Running, Role: LEADER
W20260807 22:35:55.897667 6438 leader_election.cc:334] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739): Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
I20260807 22:35:55.897742 7230 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } }
I20260807 22:35:55.898095 7235 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6b627b3aca44300b46e08e129409427. Latest consensus state: current_term: 3 leader_uuid: "f6b627b3aca44300b46e08e129409427" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:55.898181 7235 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:55.898356 7241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:55.898808 7241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:35:55.898869 7241 catalog_manager.cc:1259] Loaded cluster ID: 951f417b1e7f44d5b8886c82cc9e9e6e
I20260807 22:35:55.898916 7241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:55.899108 7241 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:55.899196 7241 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Loaded TSK: 0
I20260807 22:35:55.899351 7241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:35:55.959174 6454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:55.960769 6454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:55.961757 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.961626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58696)
0807 22:35:55.961681 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.961723 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 22:35:55.961794 6203 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [term 3 FOLLOWER]: Refusing update from remote peer f6b627b3aca44300b46e08e129409427: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 22:35:55.962020 7235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 22:35:55.962284 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.962105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.0.0.1:58710)
0807 22:35:55.962223 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.962257 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260807 22:35:55.962368 6438 consensus_peers.cc:487] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 -> Peer 21af200769d44c058aa2fb4efd2fd9a2 (127.28.82.254:36739): Couldn't send request to peer 21af200769d44c058aa2fb4efd2fd9a2. Status: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:35:55.962766 7245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6b627b3aca44300b46e08e129409427. Latest consensus state: current_term: 3 leader_uuid: "f6b627b3aca44300b46e08e129409427" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:55.962880 7245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:35:55.963299 7230 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f6b627b3aca44300b46e08e129409427" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:55.963389 7230 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:55.963812 7245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f6b627b3aca44300b46e08e129409427" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "21af200769d44c058aa2fb4efd2fd9a2" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36739 } } peers { permanent_uuid: "20c8d9158ef54b8daf236ea65cafaf55" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35269 } attrs { promote: false } } peers { permanent_uuid: "f6b627b3aca44300b46e08e129409427" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 44363 } attrs { promote: false } } }
I20260807 22:35:55.964013 7245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c8d9158ef54b8daf236ea65cafaf55 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:35:55.968367 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.965074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:38485 (local address 127.0.0.1:36734)
0807 22:35:55.965105 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.965118 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:55.965174 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:55.965517 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:55.965518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:55.965520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:55.965583 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:55.965585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.966598 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.966599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:55.967391 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:55.967394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.968141 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.968142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:55.968220 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:55.968226 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:35:55.968254 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 22:35:55.968297 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":3}
I20260807 22:35:55.968751 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.965200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.28.82.193:38485)
0807 22:35:55.965362 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:55.965365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:55.965374 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:35:55.965402 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:55.965403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:55.965416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:55.965653 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:55.965700 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.966505 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.967518 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.968044 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.968577 (+ 533us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:55.968592 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:55.968611 (+ 19us) server_negotiation.cc:299] Negotiation successful
0807 22:35:55.968651 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260807 22:35:55.968875 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.965165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:37031 (local address 127.0.0.1:55096)
0807 22:35:55.965306 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.965317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:55.965416 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:55.965816 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:55.965817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:55.965819 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:55.965867 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:55.965868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.966705 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.966706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:55.967850 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:55.967853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.968622 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.968623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:55.968687 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:55.968689 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:35:55.968747 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 22:35:55.968793 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":12,"thread_start_us":80,"threads_started":1}
I20260807 22:35:55.968897 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.965528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.28.82.194:37031)
0807 22:35:55.965650 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:55.965652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:55.965660 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:55.965686 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:55.965687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:55.965698 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:55.965805 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:55.965937 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.966646 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.967938 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.968518 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.968620 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:55.968631 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:55.968763 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 22:35:55.968807 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260807 22:35:55.969132 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.965445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59862 (local address 127.28.82.195:38963)
0807 22:35:55.965704 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:55.965707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:55.965715 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:55.965741 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:55.965742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:55.965775 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:55.965840 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:55.965984 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.966544 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.968003 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.968390 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.968562 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:55.968577 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:55.969011 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 22:35:55.969065 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":53,"threads_started":1}
I20260807 22:35:55.969122 6612 tablet_service.cc:1467] Processing CreateTablet for tablet a7da5c797c4a4b9caef68d8611ddb821 (DEFAULT_TABLE table=test-table [id=66083af3c69846dea1e9b3c54209bee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:55.969147 6816 tablet_service.cc:1467] Processing CreateTablet for tablet a7da5c797c4a4b9caef68d8611ddb821 (DEFAULT_TABLE table=test-table [id=66083af3c69846dea1e9b3c54209bee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:55.969269 6816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7da5c797c4a4b9caef68d8611ddb821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:55.969269 6612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7da5c797c4a4b9caef68d8611ddb821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:55.970868 7263 tablet_bootstrap.cc:492] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793: Bootstrap starting.
I20260807 22:35:55.970868 7261 tablet_bootstrap.cc:492] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: Bootstrap starting.
I20260807 22:35:55.971050 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.965349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38963 (local address 127.0.0.1:59862)
0807 22:35:55.965552 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.965563 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:55.965612 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:55.965885 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:55.965886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:55.965888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:55.965921 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:55.965923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.966639 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.966640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:55.967943 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:55.967944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:55.968612 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:55.968613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:55.968684 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:55.968688 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:55.968747 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:35:55.968793 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":46,"thread_start_us":73,"threads_started":1}
I20260807 22:35:55.971280 7261 tablet_bootstrap.cc:654] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:55.971280 7263 tablet_bootstrap.cc:654] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:55.971340 7013 tablet_service.cc:1467] Processing CreateTablet for tablet a7da5c797c4a4b9caef68d8611ddb821 (DEFAULT_TABLE table=test-table [id=66083af3c69846dea1e9b3c54209bee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:55.971468 7013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7da5c797c4a4b9caef68d8611ddb821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:55.971884 7263 tablet_bootstrap.cc:492] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793: No bootstrap required, opened a new log
I20260807 22:35:55.971930 7263 ts_tablet_manager.cc:1397] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:55.972014 7263 raft_consensus.cc:348] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:55.972067 7263 raft_consensus.cc:374] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:55.972081 7263 raft_consensus.cc:729] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b06e445ec1c4b1496ae351e79d8f793, State: Initialized, Role: FOLLOWER
I20260807 22:35:55.972141 7263 consensus_queue.cc:260] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:55.972343 7263 ts_tablet_manager.cc:1428] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:55.972631 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.972489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:59785)
0807 22:35:55.972557 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.972594 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 22:35:55.973358 7261 tablet_bootstrap.cc:492] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: No bootstrap required, opened a new log
I20260807 22:35:55.973371 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.973250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:47145)
0807 22:35:55.973311 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.973340 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 22:35:55.973423 7261 ts_tablet_manager.cc:1397] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 22:35:55.973551 7261 raft_consensus.cc:348] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:55.973652 7261 raft_consensus.cc:374] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:55.973399 7267 tablet_bootstrap.cc:492] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Bootstrap starting.
I20260807 22:35:55.973685 7261 raft_consensus.cc:729] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1038c95d4b5445c88e918ea5fcf5ffdc, State: Initialized, Role: FOLLOWER
I20260807 22:35:55.973733 7261 consensus_queue.cc:260] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:55.973922 7261 ts_tablet_manager.cc:1428] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:55.974289 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.974184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:43905)
0807 22:35:55.974223 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.974256 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260807 22:35:55.974324 7267 tablet_bootstrap.cc:654] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:55.974385 6918 heartbeater.cc:503] Master 127.28.82.252:44363 was elected leader, sending a full tablet report...
I20260807 22:35:55.974828 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.974731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:39547)
0807 22:35:55.974767 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.974794 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 22:35:55.975103 7267 tablet_bootstrap.cc:492] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: No bootstrap required, opened a new log
I20260807 22:35:55.975145 7267 ts_tablet_manager.cc:1397] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 22:35:55.975241 7267 raft_consensus.cc:348] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:55.975291 7267 raft_consensus.cc:374] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:55.975303 7267 raft_consensus.cc:729] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d66649e0dd4ce1bd1eea96ddf2c74c, State: Initialized, Role: FOLLOWER
I20260807 22:35:55.975364 7267 consensus_queue.cc:260] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:55.975536 7267 ts_tablet_manager.cc:1428] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:55.975852 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.975742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:40559)
0807 22:35:55.975792 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.975820 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 22:35:55.975986 7116 heartbeater.cc:503] Master 127.28.82.252:44363 was elected leader, sending a full tablet report...
I20260807 22:35:55.976516 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:55.976402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:60557)
0807 22:35:55.976435 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:55.976469 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260807 22:35:56.097837 6713 tablet.cc:2369] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:35:56.137455 7119 tablet.cc:2369] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:56.255368 7272 raft_consensus.cc:482] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:56.255476 7272 raft_consensus.cc:504] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:56.255797 7272 leader_election.cc:288] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b06e445ec1c4b1496ae351e79d8f793 (127.28.82.193:38485), 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031)
I20260807 22:35:56.260068 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.256095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:58343 (local address 127.28.82.193:38485)
0807 22:35:56.256175 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:56.256178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:56.256190 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:35:56.256295 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:56.256297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:56.256310 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:56.256380 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:56.256551 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.257361 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.259059 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.259608 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.259692 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:56.259702 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:56.259953 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 22:35:56.259995 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 22:35:56.260068 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.256253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:52069 (local address 127.28.82.194:37031)
0807 22:35:56.256300 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:56.256302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:56.256451 (+ 149us) server_negotiation.cc:407] Connection header received
0807 22:35:56.256479 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:56.256480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:56.256493 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:56.256552 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:56.256708 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.257397 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.259060 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.259608 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.259692 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:56.259710 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:56.259953 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 22:35:56.259995 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 22:35:56.260103 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.256039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:38485 (local address 127.28.82.195:58343)
0807 22:35:56.256091 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.256106 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:56.256204 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:56.256401 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:56.256403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:56.256405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:56.256480 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:56.256482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.257470 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.257472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:56.258953 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:56.258955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.259757 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.259760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:56.259851 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:56.259855 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:56.259933 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 22:35:56.259989 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":144}
I20260807 22:35:56.260123 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.256148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:37031 (local address 127.28.82.195:52069)
0807 22:35:56.256352 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.256364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:56.256422 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:56.256572 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:56.256573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:56.256574 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:56.256628 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:56.256629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.257470 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.257472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:56.258948 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:56.258951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.259775 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.259776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:56.259849 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:56.259854 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:35:56.259933 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:35:56.259989 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":101,"thread_start_us":151,"threads_started":1}
I20260807 22:35:56.260442 6846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7da5c797c4a4b9caef68d8611ddb821" candidate_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" is_pre_election: true
I20260807 22:35:56.260455 6643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7da5c797c4a4b9caef68d8611ddb821" candidate_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b06e445ec1c4b1496ae351e79d8f793" is_pre_election: true
I20260807 22:35:56.260567 6846 raft_consensus.cc:2454] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0d66649e0dd4ce1bd1eea96ddf2c74c in term 0.
I20260807 22:35:56.260603 6643 raft_consensus.cc:2454] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0d66649e0dd4ce1bd1eea96ddf2c74c in term 0.
I20260807 22:35:56.260862 6953 leader_election.cc:302] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b06e445ec1c4b1496ae351e79d8f793, e0d66649e0dd4ce1bd1eea96ddf2c74c; no voters:
I20260807 22:35:56.261011 7272 raft_consensus.cc:2781] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:56.261060 7272 raft_consensus.cc:482] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:56.261091 7272 raft_consensus.cc:3037] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:56.261811 7272 raft_consensus.cc:504] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:56.261955 7272 leader_election.cc:288] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [CANDIDATE]: Term 1 election: Requested vote from peers 8b06e445ec1c4b1496ae351e79d8f793 (127.28.82.193:38485), 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031)
I20260807 22:35:56.262184 6643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7da5c797c4a4b9caef68d8611ddb821" candidate_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b06e445ec1c4b1496ae351e79d8f793"
I20260807 22:35:56.262207 6846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7da5c797c4a4b9caef68d8611ddb821" candidate_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc"
I20260807 22:35:56.262282 6643 raft_consensus.cc:3037] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:56.262297 6846 raft_consensus.cc:3037] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:56.263060 6643 raft_consensus.cc:2454] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0d66649e0dd4ce1bd1eea96ddf2c74c in term 1.
I20260807 22:35:56.263108 6846 raft_consensus.cc:2454] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0d66649e0dd4ce1bd1eea96ddf2c74c in term 1.
I20260807 22:35:56.263304 6953 leader_election.cc:302] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1038c95d4b5445c88e918ea5fcf5ffdc, e0d66649e0dd4ce1bd1eea96ddf2c74c; no voters:
I20260807 22:35:56.263454 7272 raft_consensus.cc:2781] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:56.263671 7272 raft_consensus.cc:686] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 LEADER]: Becoming Leader. State: Replica: e0d66649e0dd4ce1bd1eea96ddf2c74c, State: Running, Role: LEADER
I20260807 22:35:56.263764 7272 consensus_queue.cc:237] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } }
I20260807 22:35:56.264048 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.263921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:43303)
0807 22:35:56.263976 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.264013 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 22:35:56.264634 6455 catalog_manager.cc:5495] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c reported cstate change: term changed from 0 to 1, leader changed from <none> to e0d66649e0dd4ce1bd1eea96ddf2c74c (127.28.82.195). New cstate: current_term: 1 leader_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0d66649e0dd4ce1bd1eea96ddf2c74c" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38963 } health_report { overall_health: HEALTHY } } }
I20260807 22:35:56.321774 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.319248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59876 (local address 127.28.82.195:38963)
0807 22:35:56.319320 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:56.319323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:56.319332 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:35:56.319385 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:56.319386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:56.319413 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:56.319495 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:56.319642 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.320128 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.320885 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:56.321103 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:56.321214 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:56.321380 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:56.321513 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:56.321616 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:56.321657 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 22:35:56.321680 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:35:56.323040 6846 raft_consensus.cc:1261] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 1 FOLLOWER]: Refusing update from remote peer e0d66649e0dd4ce1bd1eea96ddf2c74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:56.323206 6643 raft_consensus.cc:1261] T a7da5c797c4a4b9caef68d8611ddb821 P 8b06e445ec1c4b1496ae351e79d8f793 [term 1 FOLLOWER]: Refusing update from remote peer e0d66649e0dd4ce1bd1eea96ddf2c74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:56.323367 7274 consensus_queue.cc:1035] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1038c95d4b5445c88e918ea5fcf5ffdc" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:56.323491 7272 consensus_queue.cc:1035] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b06e445ec1c4b1496ae351e79d8f793" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:56.324214 6709 heartbeater.cc:503] Master 127.28.82.252:44363 was elected leader, sending a full tablet report...
I20260807 22:35:56.324465 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.324160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:44367)
0807 22:35:56.324392 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.324430 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260807 22:35:56.324628 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.324458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:50661)
0807 22:35:56.324560 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.324597 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 22:35:56.325671 7278 mvcc.cc:205] Tried to move back new op lower bound from 7316038272296534016 to 7316038272057118720. Current Snapshot: MvccSnapshot[applied={T|T < 7316038272296534016}]
I20260807 22:35:56.326228 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.326122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.194:51305)
0807 22:35:56.326163 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.326196 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 22:35:56.326578 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.326446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:54555)
0807 22:35:56.326509 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.326541 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 22:35:56.327180 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.327051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.193:42279)
0807 22:35:56.327121 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.327149 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 22:35:56.327524 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.326852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36739 (local address 127.28.82.195:33309)
0807 22:35:56.327469 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.327498 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602}
I20260807 22:35:56.328413 6454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:56.328502 6454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:56.329813 6454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6b627b3aca44300b46e08e129409427: Sending DeleteTablet for 3 replicas of tablet a7da5c797c4a4b9caef68d8611ddb821
I20260807 22:35:56.330209 6816 tablet_service.cc:1514] Processing DeleteTablet for tablet a7da5c797c4a4b9caef68d8611ddb821 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:56 UTC) from {username='oryx', principal='oryx/127.28.82.252@KRBTEST.COM'} at 127.0.0.1:55096
I20260807 22:35:56.330206 7013 tablet_service.cc:1514] Processing DeleteTablet for tablet a7da5c797c4a4b9caef68d8611ddb821 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:56 UTC) from {username='oryx', principal='oryx/127.28.82.252@KRBTEST.COM'} at 127.0.0.1:59862
I20260807 22:35:56.330432 6612 tablet_service.cc:1514] Processing DeleteTablet for tablet a7da5c797c4a4b9caef68d8611ddb821 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:56 UTC) from {username='oryx', principal='oryx/127.28.82.252@KRBTEST.COM'} at 127.0.0.1:36734
I20260807 22:35:56.330549 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 6393
I20260807 22:35:56.338570 7291 tablet_replica.cc:330] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: stopping tablet replica
I20260807 22:35:56.338655 7291 raft_consensus.cc:2229] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:56.338709 7291 raft_consensus.cc:2258] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:56.338853 7292 tablet_replica.cc:330] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: stopping tablet replica
I20260807 22:35:56.338904 7292 raft_consensus.cc:2229] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:56.339006 7292 raft_consensus.cc:2258] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:56.339293 7292 ts_tablet_manager.cc:1905] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:56.342196 7292 ts_tablet_manager.cc:1918] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:56.342290 7292 log.cc:1196] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/a7da5c797c4a4b9caef68d8611ddb821
I20260807 22:35:56.342592 7292 ts_tablet_manager.cc:1939] T a7da5c797c4a4b9caef68d8611ddb821 P e0d66649e0dd4ce1bd1eea96ddf2c74c: Deleting consensus metadata
I20260807 22:35:56.342955 6438 catalog_manager.cc:4841] TS e0d66649e0dd4ce1bd1eea96ddf2c74c (127.28.82.195:38963): tablet a7da5c797c4a4b9caef68d8611ddb821 (table test-table [id=66083af3c69846dea1e9b3c54209bee1]) successfully deleted
I20260807 22:35:56.343796 7291 ts_tablet_manager.cc:1905] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:56.345067 7291 ts_tablet_manager.cc:1918] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:56.345151 7291 log.cc:1196] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/a7da5c797c4a4b9caef68d8611ddb821
I20260807 22:35:56.345458 7291 ts_tablet_manager.cc:1939] T a7da5c797c4a4b9caef68d8611ddb821 P 1038c95d4b5445c88e918ea5fcf5ffdc: Deleting consensus metadata
I20260807 22:35:56.345778 6438 catalog_manager.cc:4841] TS 1038c95d4b5445c88e918ea5fcf5ffdc (127.28.82.194:37031): tablet a7da5c797c4a4b9caef68d8611ddb821 (table test-table [id=66083af3c69846dea1e9b3c54209bee1]) successfully deleted
I20260807 22:35:56.351629 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 6726
I20260807 22:35:56.351701 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:56.351547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:38485 (local address 127.0.0.1:36748)
0807 22:35:56.351611 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:56.351655 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:38485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:35:56.354238 6438 catalog_manager.cc:4584] TS 8b06e445ec1c4b1496ae351e79d8f793 (127.28.82.193:38485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7da5c797c4a4b9caef68d8611ddb821: Network error: Client connection negotiation failed: client connection to 127.28.82.193:38485: connect: Connection refused (error 111)
I20260807 22:35:56.359390 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 6926
I20260807 22:35:56.368459 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 5997
I20260807 22:35:56.381104 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 6265
2026-08-07T22:35:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7063 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:56 dist-test-slave-7kzw krb5kdc[7298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:56 dist-test-slave-7kzw krb5kdc[7298](info): set up 2 sockets
Aug 07 22:35:56 dist-test-slave-7kzw krb5kdc[7298](info): commencing operation
krb5kdc: starting...
W20260807 22:35:57.602872 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:35:57 dist-test-slave-7kzw krb5kdc[7298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:35:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:35:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.254.keytab.
Entry for principal oryx/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.254.keytab.
I20260807 22:35:57.737522 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:44643
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.254.keytab
--principal=oryx/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:34161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:44643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:57.820364 7316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:57.820619 7316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:57.820655 7316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:57.821775 7316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:35:57.821846 7316 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 22:35:57.821882 7316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:57.821918 7316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:57.821946 7316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:35:57.821970 7316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:35:57.823530 7316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:34161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:44643
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.254.keytab
--principal=oryx/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:44643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:57.823822 7316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:57.824050 7316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:57.829020 7327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:57.829066 7326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:57.829223 7329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:57.829519 7316 server_base.cc:1034] running on GCE node
I20260807 22:35:57.829790 7316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:57.830034 7316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:57.831173 7316 hybrid_clock.cc:648] HybridClock initialized: now 1786142157831157 us; error 32 us; skew 500 ppm
Aug 07 22:35:57 dist-test-slave-7kzw krb5kdc[7298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142157, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:57.833655 7316 init.cc:377] Logged in from keytab as oryx/127.28.82.254@KRBTEST.COM (short username oryx)
I20260807 22:35:57.834128 7316 webserver.cc:460] Webserver started at http://127.28.82.254:34123/ using document root <none> and password file <none>
I20260807 22:35:57.834372 7316 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:57.834419 7316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:57.834592 7316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:57.835659 7316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "656d04a541c846db8a8e03d0d1be5478"
format_stamp: "Formatted at 2026-08-07 22:35:57 on dist-test-slave-7kzw"
server_key: "f9c258fdef35b5c137f96c4e50ccf580"
server_key_iv: "079edd7914ed4abce49ef3623689a14b"
server_key_version: "encryptionkey@0"
I20260807 22:35:57.836042 7316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "656d04a541c846db8a8e03d0d1be5478"
format_stamp: "Formatted at 2026-08-07 22:35:57 on dist-test-slave-7kzw"
server_key: "f9c258fdef35b5c137f96c4e50ccf580"
server_key_iv: "079edd7914ed4abce49ef3623689a14b"
server_key_version: "encryptionkey@0"
I20260807 22:35:57.837481 7316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:57.838235 7342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:57.838411 7316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:57.838536 7316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "656d04a541c846db8a8e03d0d1be5478"
format_stamp: "Formatted at 2026-08-07 22:35:57 on dist-test-slave-7kzw"
server_key: "f9c258fdef35b5c137f96c4e50ccf580"
server_key_iv: "079edd7914ed4abce49ef3623689a14b"
server_key_version: "encryptionkey@0"
I20260807 22:35:57.838618 7316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:57.851372 7316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:57.853319 7316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:57.853484 7316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:57.858366 7316 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:44643
I20260807 22:35:57.858460 7429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:44643 every 8 connection(s)
I20260807 22:35:57.858830 7316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:35:57.859524 7430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:57.861868 7430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: Bootstrap starting.
I20260807 22:35:57.862509 7430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:57.862834 7430 log.cc:824] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:57.862942 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 7316
I20260807 22:35:57.863054 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:35:57.863312 29003 external_mini_cluster.cc:1428] Setting key d3e872d7c51f9feb1dd346647ae6dfaa
I20260807 22:35:57.863727 7430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: No bootstrap required, opened a new log
I20260807 22:35:57.866808 7430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656d04a541c846db8a8e03d0d1be5478" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44643 } }
I20260807 22:35:57.866952 7430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:57.866977 7430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 656d04a541c846db8a8e03d0d1be5478, State: Initialized, Role: FOLLOWER
I20260807 22:35:57.867136 7430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656d04a541c846db8a8e03d0d1be5478" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44643 } }
I20260807 22:35:57.867211 7430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:35:57.867256 7430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:35:57.867323 7430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:57.868072 7430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656d04a541c846db8a8e03d0d1be5478" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44643 } }
I20260807 22:35:57.868192 7430 leader_election.cc:302] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 656d04a541c846db8a8e03d0d1be5478; no voters:
I20260807 22:35:57.868366 7430 leader_election.cc:288] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:35:57.868428 7436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:57.868615 7430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:35:57.868628 7436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [term 1 LEADER]: Becoming Leader. State: Replica: 656d04a541c846db8a8e03d0d1be5478, State: Running, Role: LEADER
I20260807 22:35:57.869583 7436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656d04a541c846db8a8e03d0d1be5478" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44643 } }
I20260807 22:35:57.870282 7437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 656d04a541c846db8a8e03d0d1be5478. Latest consensus state: current_term: 1 leader_uuid: "656d04a541c846db8a8e03d0d1be5478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656d04a541c846db8a8e03d0d1be5478" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44643 } } }
I20260807 22:35:57.870416 7437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [sys.catalog]: This master's current role is: LEADER
I20260807 22:35:57.871304 7440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "656d04a541c846db8a8e03d0d1be5478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656d04a541c846db8a8e03d0d1be5478" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 44643 } } }
W20260807 22:35:57.871315 7455 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 22:35:57.871387 7440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478 [sys.catalog]: This master's current role is: LEADER
W20260807 22:35:57.871397 7455 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:35:57.871510 7456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:35:57.871713 7456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 22:35:57 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:57.873997 7456 catalog_manager.cc:1347] Generated new cluster ID: dfb86ba3ef1e4e85ad7cb393328eeaf8
I20260807 22:35:57.874060 7456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:35:57.876842 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:57.863871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.28.82.254:44643)
0807 22:35:57.864041 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:57.864050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:57.864087 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:35:57.865084 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:57.865092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:57.865144 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:57.865273 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:57.866320 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:57.867508 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:57.868730 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:57.869296 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:57.869429 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:57.872638 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:57.872642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:57.872642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:57.872655 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:57.875038 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:57.875551 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:57.875554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:57.875554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:57.875666 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:57.875902 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:57.875903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:57.875903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:57.876210 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:57.876359 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:57.876492 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 22:35:57.876626 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260807 22:35:57.912206 7456 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:35:57.912866 7456 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:35:57.915753 7456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: Generated new TSK 0
I20260807 22:35:57.915959 7456 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab.
Entry for principal oryx/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab.
I20260807 22:35:57.945051 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44643
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab
--principal=oryx/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:34161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:58.028267 7464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:58.028513 7464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:58.028555 7464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:58.029692 7464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:58.029772 7464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:58.029811 7464 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:58.029847 7464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:35:58.031427 7464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:34161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab
--principal=oryx/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:44643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:58.031695 7464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:58.031939 7464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:58.037052 7477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:58.037052 7474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:58.037261 7464 server_base.cc:1034] running on GCE node
W20260807 22:35:58.037398 7475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:58.037621 7464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:58.037901 7464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:58.039067 7464 hybrid_clock.cc:648] HybridClock initialized: now 1786142158039062 us; error 37 us; skew 500 ppm
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142158, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:58.041704 7464 init.cc:377] Logged in from keytab as oryx/127.28.82.193@KRBTEST.COM (short username oryx)
I20260807 22:35:58.042152 7464 webserver.cc:460] Webserver started at http://127.28.82.193:43137/ using document root <none> and password file <none>
I20260807 22:35:58.042426 7464 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:58.042526 7464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:58.042683 7464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:58.043710 7464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "4080b3e078b5401aa3c45a8dceb41f45"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "595971b88a7ffc31da9f581ca9fba692"
server_key_iv: "3df215582bd3b4bc2868754bfe294d6f"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.044092 7464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "4080b3e078b5401aa3c45a8dceb41f45"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "595971b88a7ffc31da9f581ca9fba692"
server_key_iv: "3df215582bd3b4bc2868754bfe294d6f"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.045444 7464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:58.046204 7489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.046391 7464 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:58.046514 7464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "4080b3e078b5401aa3c45a8dceb41f45"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "595971b88a7ffc31da9f581ca9fba692"
server_key_iv: "3df215582bd3b4bc2868754bfe294d6f"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.046608 7464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:58.071620 7464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:58.073521 7464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:58.073697 7464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:58.074235 7464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:58.074306 7464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.074359 7464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:58.074383 7464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142158, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:58.081432 7464 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:42363
I20260807 22:35:58.081511 7648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:42363 every 8 connection(s)
I20260807 22:35:58.081887 7464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:35:58.084501 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.074797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:47589 (local address 127.28.82.254:44643)
0807 22:35:58.074873 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.074875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.075130 (+ 255us) server_negotiation.cc:407] Connection header received
0807 22:35:58.075634 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.075636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.075650 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.075732 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.076667 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.077514 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.078925 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.079163 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.079245 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.081685 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:58.081686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:58.081687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:58.081696 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:58.083309 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.083672 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.083673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.083673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.083728 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.083912 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.083913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.083913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.084034 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:58.084086 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.084352 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.084431 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:35:58.084638 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.074793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44643 (local address 127.28.82.193:47589)
0807 22:35:58.074995 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.075035 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.075535 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.075845 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.075849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.076238 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:58.076554 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.076562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.077633 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.077634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.078826 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.078830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.079289 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.079291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.079415 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.079932 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:58.079941 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:58.081551 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:58.083383 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.083388 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.083390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.083603 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.083777 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.083778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.083778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.083861 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.084099 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:58.084105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:58.084236 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.084322 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.084446 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":135,"threads_started":1}
I20260807 22:35:58.085124 7649 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44643
I20260807 22:35:58.085212 7649 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:58.085414 7649 heartbeater.cc:511] Master 127.28.82.254:44643 requested a full tablet report, sending...
I20260807 22:35:58.086033 7362 ts_manager.cc:194] Registered new tserver with Master: 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193:42363)
I20260807 22:35:58.087011 7362 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:47589
I20260807 22:35:58.090869 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 7464
I20260807 22:35:58.090986 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:35:58.091264 29003 external_mini_cluster.cc:1428] Setting key 73735b92a055d61bf0b5723683d18cb8
I20260807 22:35:58.091964 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.086270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:43883 (local address 127.28.82.254:44643)
0807 22:35:58.086315 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.086317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.086459 (+ 142us) server_negotiation.cc:407] Connection header received
0807 22:35:58.086522 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.086523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.086532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.086585 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.086908 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.087454 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.088337 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.088512 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.088561 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.089444 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:58.089445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:58.089445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:58.089450 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:58.090653 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.091104 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.091105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.091106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.091159 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.091400 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.091401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.091401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.091542 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:58.091610 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.091797 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.091872 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:35:58.091979 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.086206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44643 (local address 127.28.82.193:43883)
0807 22:35:58.086382 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.086393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.086457 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.086600 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.086601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.086794 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:58.086848 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.086850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.087578 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.087579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.088265 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.088268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.088575 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.088576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.088623 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.089066 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:58.089072 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:58.089376 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:58.090733 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.090734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.090734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.090989 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.091227 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.091228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.091228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.091301 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.091647 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:58.091648 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:58.091685 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.091771 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.091874 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":31,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for oryx/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab.
Entry for principal oryx/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab.
I20260807 22:35:58.117547 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44643
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab
--principal=oryx/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:34161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:58.200134 7662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:58.200342 7662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:58.200366 7662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:58.201620 7662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:58.201665 7662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:58.201686 7662 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:58.201704 7662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:35:58.203202 7662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:34161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab
--principal=oryx/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:44643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:58.203444 7662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:58.203696 7662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:35:58.208997 7662 server_base.cc:1034] running on GCE node
W20260807 22:35:58.208981 7670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:58.208997 7672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:58.208981 7669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:58.209434 7662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:58.209699 7662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:58.210865 7662 hybrid_clock.cc:648] HybridClock initialized: now 1786142158210855 us; error 30 us; skew 500 ppm
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142158, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:58.213527 7662 init.cc:377] Logged in from keytab as oryx/127.28.82.194@KRBTEST.COM (short username oryx)
I20260807 22:35:58.214011 7662 webserver.cc:460] Webserver started at http://127.28.82.194:42041/ using document root <none> and password file <none>
I20260807 22:35:58.214243 7662 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:58.214318 7662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:58.214514 7662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:58.215540 7662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "316d25c0c023480fb16339ad4e7b54cb"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "a10d653c75a27a51a9a3179724f3f0fe"
server_key_iv: "bcffdf30f56b3efd18d04c79cce31493"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.215915 7662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "316d25c0c023480fb16339ad4e7b54cb"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "a10d653c75a27a51a9a3179724f3f0fe"
server_key_iv: "bcffdf30f56b3efd18d04c79cce31493"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.217271 7662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:35:58.217978 7681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.218154 7662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:58.218262 7662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "316d25c0c023480fb16339ad4e7b54cb"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "a10d653c75a27a51a9a3179724f3f0fe"
server_key_iv: "bcffdf30f56b3efd18d04c79cce31493"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.218338 7662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:58.227782 7662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:58.230077 7662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:58.230242 7662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:58.230866 7662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:58.230926 7662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.230960 7662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:58.230988 7662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142158, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:58.238533 7662 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:46181
I20260807 22:35:58.238601 7844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:46181 every 8 connection(s)
I20260807 22:35:58.238936 7662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:35:58.241029 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.231218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:60265 (local address 127.28.82.254:44643)
0807 22:35:58.231303 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.231306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.231461 (+ 155us) server_negotiation.cc:407] Connection header received
0807 22:35:58.231927 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.231928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.231943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.232026 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.232691 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.233541 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.234586 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.234928 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.235042 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.238212 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:58.238214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:58.238214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:58.238224 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:58.239816 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.240227 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.240227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.240228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.240285 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.240446 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.240447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.240447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.240557 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:58.240636 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.240881 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.240961 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 22:35:58.241194 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.231172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44643 (local address 127.28.82.194:60265)
0807 22:35:58.231317 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.231356 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.231819 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.232079 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.232082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.232374 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:58.232593 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.232597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.233629 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.233630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.234456 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.234458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.235039 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.235040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.235192 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.235706 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:58.235716 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:58.238110 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:58.239893 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.239919 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.239922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.240172 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.240330 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.240330 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.240331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.240412 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.240631 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:58.240638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:58.240760 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.240847 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.240999 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260807 22:35:58.241715 7845 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44643
I20260807 22:35:58.241837 7845 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:58.242014 7845 heartbeater.cc:511] Master 127.28.82.254:44643 requested a full tablet report, sending...
I20260807 22:35:58.242518 7362 ts_manager.cc:194] Registered new tserver with Master: 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181)
I20260807 22:35:58.242623 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 7662
I20260807 22:35:58.242740 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:35:58.242944 29003 external_mini_cluster.cc:1428] Setting key 8b274f165f88507b83893dbd0ed9dad4
I20260807 22:35:58.243049 7362 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:60265
I20260807 22:35:58.248792 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.242790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:41825 (local address 127.28.82.254:44643)
0807 22:35:58.242830 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.242832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.242961 (+ 129us) server_negotiation.cc:407] Connection header received
0807 22:35:58.243031 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.243032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.243043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.243106 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.243717 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.244173 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.244866 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.245081 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.245156 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.246153 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:58.246154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:58.246154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:58.246160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:58.247599 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.248020 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.248022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.248022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.248088 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.248310 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.248311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.248311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.248444 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:58.248497 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.248670 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.248727 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:35:58.248960 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.242745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44643 (local address 127.28.82.194:41825)
0807 22:35:58.242883 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.242893 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.242975 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.243111 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.243112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.243557 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:58.243616 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.243618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.244272 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.244272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.244781 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.244782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.245217 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.245218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.245293 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.245659 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:58.245665 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:58.246054 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:58.247686 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.247687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.247687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.247933 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.248163 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.248164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.248164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.248241 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.248513 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:58.248514 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:58.248566 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.248644 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.248754 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab.
Entry for principal oryx/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab.
I20260807 22:35:58.273162 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:44643
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab
--principal=oryx/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:34161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:35:58.356626 7858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:35:58.356855 7858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:35:58.356892 7858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:35:58.358021 7858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:35:58.358091 7858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:35:58.358124 7858 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 22:35:58.358170 7858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:35:58.359755 7858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:34161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab
--principal=oryx/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.82.254:44643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:35:58.360030 7858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:35:58.360275 7858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:35:58.365415 7868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:58.365415 7872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:35:58.365494 7867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:35:58.365466 7858 server_base.cc:1034] running on GCE node
I20260807 22:35:58.365994 7858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:35:58.366235 7858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:35:58.367372 7858 hybrid_clock.cc:648] HybridClock initialized: now 1786142158367364 us; error 55 us; skew 500 ppm
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142158, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:35:58.370186 7858 init.cc:377] Logged in from keytab as oryx/127.28.82.195@KRBTEST.COM (short username oryx)
I20260807 22:35:58.370744 7858 webserver.cc:460] Webserver started at http://127.28.82.195:41859/ using document root <none> and password file <none>
I20260807 22:35:58.371021 7858 fs_manager.cc:359] Metadata directory not provided
I20260807 22:35:58.371083 7858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:35:58.371179 7858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:35:58.372182 7858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "5bbb4315bd6c411e867600e8a62f62ac"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "86005ff173a58d2bda7723102db90404"
server_key_iv: "95a408ff0e2b83b2bd5d8f91de81ac7e"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.372473 7858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "5bbb4315bd6c411e867600e8a62f62ac"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "86005ff173a58d2bda7723102db90404"
server_key_iv: "95a408ff0e2b83b2bd5d8f91de81ac7e"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.373812 7858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:35:58.374652 7883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.374846 7858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:58.374946 7858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "5bbb4315bd6c411e867600e8a62f62ac"
format_stamp: "Formatted at 2026-08-07 22:35:58 on dist-test-slave-7kzw"
server_key: "86005ff173a58d2bda7723102db90404"
server_key_iv: "95a408ff0e2b83b2bd5d8f91de81ac7e"
server_key_version: "encryptionkey@0"
I20260807 22:35:58.375023 7858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:35:58.394323 7858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:35:58.396375 7858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:35:58.396551 7858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:35:58.397145 7858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:35:58.397186 7858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.397209 7858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:35:58.397222 7858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142158, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:58.404861 7858 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:42933
I20260807 22:35:58.405489 7858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:35:58.406037 8039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:42933 every 8 connection(s)
I20260807 22:35:58.408865 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 7858
I20260807 22:35:58.408897 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.397576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:37813 (local address 127.28.82.254:44643)
0807 22:35:58.397640 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.397644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.397818 (+ 174us) server_negotiation.cc:407] Connection header received
0807 22:35:58.398367 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.398368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.398382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.398457 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.399174 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.399928 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.400947 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.401163 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.401243 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.404947 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:58.404949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:58.404950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:58.404959 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:58.406939 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.407503 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.407505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.407505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.407596 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.408020 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.408022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.408023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.408213 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:58.408306 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.408683 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.408775 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:35:58.409107 7895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.397546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44643 (local address 127.28.82.195:37813)
0807 22:35:58.397705 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.397740 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.398261 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.398543 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.398547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.398891 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:58.399074 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.399079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.400022 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.400023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.400842 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.400844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.401248 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.401249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.401452 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.401990 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:58.402002 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:58.404814 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:58.407032 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.407040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.407042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.407395 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.407754 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.407756 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.407756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.407925 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.408316 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:58.408323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:58.408498 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.408655 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.408852 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260807 22:35:58.409204 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:35:58.409466 29003 external_mini_cluster.cc:1428] Setting key ac2a75db598fa701f05d093a07932e2e
I20260807 22:35:58.410866 8040 heartbeater.cc:347] Connected to a master server at 127.28.82.254:44643
I20260807 22:35:58.410976 8040 heartbeater.cc:464] Registering TS with master...
I20260807 22:35:58.411163 8040 heartbeater.cc:511] Master 127.28.82.254:44643 requested a full tablet report, sending...
I20260807 22:35:58.411535 7362 ts_manager.cc:194] Registered new tserver with Master: 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:58.412101 7362 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:37813
I20260807 22:35:58.418660 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.410993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:43685 (local address 127.28.82.254:44643)
0807 22:35:58.411034 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.411036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.411304 (+ 268us) server_negotiation.cc:407] Connection header received
0807 22:35:58.411367 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.411368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.411380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.411445 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.411854 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.412760 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.413898 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.414169 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.414227 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.415476 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:58.415478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:58.415478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:58.415486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:58.417292 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.417777 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.417778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.417779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.417863 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.418087 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.418088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.418088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.418216 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:58.418290 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.418499 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.418570 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 22:35:58.418675 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.411072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44643 (local address 127.28.82.195:43685)
0807 22:35:58.411224 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.411250 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.411314 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.411463 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.411464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.411693 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:58.411776 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.411778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.412858 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.412859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.413804 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.413807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.414257 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.414259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.414357 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.414840 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:58.414847 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:58.415340 (+ 493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:58.417363 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.417393 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.417393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.417698 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.417940 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.417941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.417942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.418012 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.418295 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:58.418296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:58.418343 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.418438 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.418568 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260807 22:35:58.421571 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142158, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:35:58.453935 7435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:58.446355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.28.82.254:44643)
0807 22:35:58.446410 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.446412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.446425 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:35:58.446530 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.446532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.446547 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.446613 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.447246 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.448274 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.449586 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.449962 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.450068 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.453830 (+ 3762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54066: BlockingRecv error: recv got EOF from 127.0.0.1:54066 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142158, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:58.464294 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.454839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.28.82.254:44643)
0807 22:35:58.454882 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.454884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.455144 (+ 260us) server_negotiation.cc:407] Connection header received
0807 22:35:58.455222 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.455223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.455233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.455314 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.455999 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.456839 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.457968 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.458336 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.458411 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.461427 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:58.461428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:58.461428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:58.461435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:58.463141 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.463557 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.463558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.463559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.463628 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.463855 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.463855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.463856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.463980 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:58.464066 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.464159 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.464221 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 22:35:58.466097 7362 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:58.467806 7362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:58.474593 8075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.470432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42363 (local address 127.0.0.1:40304)
0807 22:35:58.470897 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.470906 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.470957 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.471250 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.471253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.471256 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:58.471325 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.471326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.472290 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.472292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.473394 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.473396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.474258 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.474259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.474348 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.474363 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.474437 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.474508 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":106,"threads_started":1}
I20260807 22:35:58.474610 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.470747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.28.82.193:42363)
0807 22:35:58.470898 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.470903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.470956 (+ 53us) server_negotiation.cc:407] Connection header received
0807 22:35:58.471028 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.471050 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.471068 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.471156 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:58.471418 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.472203 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.473496 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.474161 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.474258 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.474340 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.474454 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.474517 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260807 22:35:58.475091 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.470432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:42933 (local address 127.0.0.1:46122)
0807 22:35:58.471182 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.471193 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.471247 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.471416 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.471417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.471419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:58.471478 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.471479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.472791 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.472792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.473848 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.473851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.474845 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.474846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.474909 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.474913 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.474958 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.474997 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":11,"thread_start_us":114,"threads_started":1}
I20260807 22:35:58.475363 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.470430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:46181 (local address 127.0.0.1:35620)
0807 22:35:58.470674 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.470719 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.470788 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.471531 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.471532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.471534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:58.471579 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.471581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.472812 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.472813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.473727 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.473729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.475117 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.475118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.475187 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.475192 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.475243 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.475282 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":158,"threads_started":1}
I20260807 22:35:58.475322 7561 tablet_service.cc:1467] Processing CreateTablet for tablet 49212783f8e448d980c3e6d7735f992f (DEFAULT_TABLE table=test-table [id=833574054697457c9805c2a77a6d9824]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:58.475715 7561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49212783f8e448d980c3e6d7735f992f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.475747 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.470946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.28.82.195:42933)
0807 22:35:58.471144 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.471148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.471252 (+ 104us) server_negotiation.cc:407] Connection header received
0807 22:35:58.471318 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.471320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.471337 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.471418 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:58.471564 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.472684 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.473933 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.474668 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.474737 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.474810 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.475285 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.475330 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260807 22:35:58.475836 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.470752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.28.82.194:46181)
0807 22:35:58.471378 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.471382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.471392 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:58.471430 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.471432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.471449 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.471535 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:58.471649 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.472700 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.473818 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.474591 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.475092 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.475656 (+ 564us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.475684 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.475724 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":85,"threads_started":1}
I20260807 22:35:58.476341 7754 tablet_service.cc:1467] Processing CreateTablet for tablet 49212783f8e448d980c3e6d7735f992f (DEFAULT_TABLE table=test-table [id=833574054697457c9805c2a77a6d9824]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:58.476341 7958 tablet_service.cc:1467] Processing CreateTablet for tablet 49212783f8e448d980c3e6d7735f992f (DEFAULT_TABLE table=test-table [id=833574054697457c9805c2a77a6d9824]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:58.476680 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49212783f8e448d980c3e6d7735f992f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.476680 7754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49212783f8e448d980c3e6d7735f992f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.479257 8081 tablet_bootstrap.cc:492] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Bootstrap starting.
I20260807 22:35:58.479869 8081 tablet_bootstrap.cc:654] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.480235 8081 log.cc:824] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:58.480383 8082 tablet_bootstrap.cc:492] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Bootstrap starting.
I20260807 22:35:58.480695 8083 tablet_bootstrap.cc:492] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Bootstrap starting.
I20260807 22:35:58.481217 8082 tablet_bootstrap.cc:654] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.481242 8083 tablet_bootstrap.cc:654] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.481477 8083 log.cc:824] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:58.481592 8082 log.cc:824] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Log is configured to *not* fsync() on all Append() calls
I20260807 22:35:58.482172 8083 tablet_bootstrap.cc:492] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: No bootstrap required, opened a new log
I20260807 22:35:58.482226 8083 ts_tablet_manager.cc:1397] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:58.482447 8082 tablet_bootstrap.cc:492] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: No bootstrap required, opened a new log
I20260807 22:35:58.482683 8082 ts_tablet_manager.cc:1397] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260807 22:35:58.482896 8081 tablet_bootstrap.cc:492] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: No bootstrap required, opened a new log
I20260807 22:35:58.482970 8081 ts_tablet_manager.cc:1397] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 22:35:58.484179 8083 raft_consensus.cc:348] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.484351 8083 raft_consensus.cc:374] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.484390 8083 raft_consensus.cc:729] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 316d25c0c023480fb16339ad4e7b54cb, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.484429 8082 raft_consensus.cc:348] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.484537 8083 consensus_queue.cc:260] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.484635 8082 raft_consensus.cc:374] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.484683 8082 raft_consensus.cc:729] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbb4315bd6c411e867600e8a62f62ac, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.484874 8082 consensus_queue.cc:260] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.484917 8083 ts_tablet_manager.cc:1428] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:58.485042 7845 heartbeater.cc:503] Master 127.28.82.254:44643 was elected leader, sending a full tablet report...
I20260807 22:35:58.485167 8082 ts_tablet_manager.cc:1428] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:35:58.485296 8040 heartbeater.cc:503] Master 127.28.82.254:44643 was elected leader, sending a full tablet report...
I20260807 22:35:58.485764 8081 raft_consensus.cc:348] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.485926 8081 raft_consensus.cc:374] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.485968 8081 raft_consensus.cc:729] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.486091 8081 consensus_queue.cc:260] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.486362 8081 ts_tablet_manager.cc:1428] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:35:58.486433 7649 heartbeater.cc:503] Master 127.28.82.254:44643 was elected leader, sending a full tablet report...
W20260807 22:35:58.489530 7846 tablet.cc:2369] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:35:58.582767 7650 tablet.cc:2369] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:58.622308 8092 raft_consensus.cc:482] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:58.622453 8092 raft_consensus.cc:504] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.622906 8092 leader_election.cc:288] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193:42363)
I20260807 22:35:58.626091 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.623288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:53901 (local address 127.28.82.193:42363)
0807 22:35:58.623349 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.623352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.623362 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:58.623394 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.623395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.623405 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.623469 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:58.623725 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.624241 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.625327 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.625662 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.625721 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.625728 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.625956 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.626016 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:35:58.626106 7895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.623117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42363 (local address 127.28.82.195:53901)
0807 22:35:58.623184 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.623198 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.623251 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.623523 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.623525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.623527 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:58.623616 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.623619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.624332 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.624333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.625195 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.625198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.625753 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.625754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.625863 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.625869 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.625933 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.625985 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 22:35:58.626120 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.623207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:54445 (local address 127.28.82.194:46181)
0807 22:35:58.623288 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.623291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.623494 (+ 203us) server_negotiation.cc:407] Connection header received
0807 22:35:58.623528 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.623529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.623548 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.623630 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:58.623827 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.624363 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.625285 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.625728 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.625809 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.625818 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.626018 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.626058 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:35:58.626338 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.623118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:46181 (local address 127.28.82.195:54445)
0807 22:35:58.623350 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.623361 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.623428 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.623639 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.623640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.623642 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:58.623746 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.623748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.624454 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.624454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.625189 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.625191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.625830 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.625831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.625902 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.625925 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.626217 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.626262 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":14,"thread_start_us":129,"threads_started":1}
I20260807 22:35:58.626617 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49212783f8e448d980c3e6d7735f992f" candidate_uuid: "5bbb4315bd6c411e867600e8a62f62ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4080b3e078b5401aa3c45a8dceb41f45" is_pre_election: true
I20260807 22:35:58.626631 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49212783f8e448d980c3e6d7735f992f" candidate_uuid: "5bbb4315bd6c411e867600e8a62f62ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb" is_pre_election: true
I20260807 22:35:58.626804 7779 raft_consensus.cc:2454] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bbb4315bd6c411e867600e8a62f62ac in term 0.
I20260807 22:35:58.626796 7591 raft_consensus.cc:2454] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bbb4315bd6c411e867600e8a62f62ac in term 0.
I20260807 22:35:58.627060 7886 leader_election.cc:302] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 316d25c0c023480fb16339ad4e7b54cb, 5bbb4315bd6c411e867600e8a62f62ac; no voters:
I20260807 22:35:58.627223 8092 raft_consensus.cc:2781] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:58.627295 8092 raft_consensus.cc:482] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:58.627313 8092 raft_consensus.cc:3037] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.628316 8092 raft_consensus.cc:504] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.628464 8092 leader_election.cc:288] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [CANDIDATE]: Term 1 election: Requested vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193:42363)
I20260807 22:35:58.628612 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49212783f8e448d980c3e6d7735f992f" candidate_uuid: "5bbb4315bd6c411e867600e8a62f62ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb"
I20260807 22:35:58.628651 7591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49212783f8e448d980c3e6d7735f992f" candidate_uuid: "5bbb4315bd6c411e867600e8a62f62ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4080b3e078b5401aa3c45a8dceb41f45"
I20260807 22:35:58.628687 7779 raft_consensus.cc:3037] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.628742 7591 raft_consensus.cc:3037] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.629421 7779 raft_consensus.cc:2454] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bbb4315bd6c411e867600e8a62f62ac in term 1.
I20260807 22:35:58.629563 7591 raft_consensus.cc:2454] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bbb4315bd6c411e867600e8a62f62ac in term 1.
I20260807 22:35:58.629681 7886 leader_election.cc:302] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 316d25c0c023480fb16339ad4e7b54cb, 5bbb4315bd6c411e867600e8a62f62ac; no voters:
I20260807 22:35:58.629765 8092 raft_consensus.cc:2781] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:58.629896 8092 raft_consensus.cc:686] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 1 LEADER]: Becoming Leader. State: Replica: 5bbb4315bd6c411e867600e8a62f62ac, State: Running, Role: LEADER
I20260807 22:35:58.630002 8092 consensus_queue.cc:237] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.630923 7363 catalog_manager.cc:5495] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195). New cstate: current_term: 1 leader_uuid: "5bbb4315bd6c411e867600e8a62f62ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } health_report { overall_health: HEALTHY } } }
W20260807 22:35:58.657305 8044 tablet.cc:2369] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:35:58.658018 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.655727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.28.82.195:42933)
0807 22:35:58.655790 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.655792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.655800 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:35:58.655835 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.655836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.655854 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.655916 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:58.656049 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.656499 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.657213 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.657463 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.657536 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.657700 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:58.657789 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:58.657847 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.657947 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.657964 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:35:58.659910 7591 raft_consensus.cc:1261] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Refusing update from remote peer 5bbb4315bd6c411e867600e8a62f62ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:58.659983 7779 raft_consensus.cc:1261] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Refusing update from remote peer 5bbb4315bd6c411e867600e8a62f62ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:58.660281 8092 consensus_queue.cc:1035] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:58.660435 8096 consensus_queue.cc:1035] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:58.662909 8100 mvcc.cc:205] Tried to move back new op lower bound from 7316038281868390400 to 7316038281749311488. Current Snapshot: MvccSnapshot[applied={T|T < 7316038281868390400}]
I20260807 22:35:58.663421 8102 mvcc.cc:205] Tried to move back new op lower bound from 7316038281868390400 to 7316038281749311488. Current Snapshot: MvccSnapshot[applied={T|T < 7316038281868390400}]
I20260807 22:35:58.663888 8101 mvcc.cc:205] Tried to move back new op lower bound from 7316038281868390400 to 7316038281749311488. Current Snapshot: MvccSnapshot[applied={T|T < 7316038281868390400}]
I20260807 22:35:58.666005 7363 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:58.666109 7363 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:58.667026 7363 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: Sending DeleteTablet for 3 replicas of tablet 49212783f8e448d980c3e6d7735f992f
I20260807 22:35:58.667395 7754 tablet_service.cc:1514] Processing DeleteTablet for tablet 49212783f8e448d980c3e6d7735f992f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:58 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:35620
I20260807 22:35:58.667412 7561 tablet_service.cc:1514] Processing DeleteTablet for tablet 49212783f8e448d980c3e6d7735f992f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:58 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40304
I20260807 22:35:58.667477 7958 tablet_service.cc:1514] Processing DeleteTablet for tablet 49212783f8e448d980c3e6d7735f992f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:58 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:46122
I20260807 22:35:58.667644 8116 tablet_replica.cc:330] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: stopping tablet replica
I20260807 22:35:58.667748 8116 raft_consensus.cc:2229] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:58.667862 8116 raft_consensus.cc:2258] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:58.667929 7363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:58.668087 7363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:58.668078 8117 tablet_replica.cc:330] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: stopping tablet replica
I20260807 22:35:58.668128 8115 tablet_replica.cc:330] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: stopping tablet replica
I20260807 22:35:58.668181 8117 raft_consensus.cc:2229] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:58.668210 8115 raft_consensus.cc:2229] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:58.668298 8117 raft_consensus.cc:2258] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:58.668308 8115 raft_consensus.cc:2258] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:58.668795 8115 ts_tablet_manager.cc:1905] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:58.669003 8117 ts_tablet_manager.cc:1905] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:58.669785 7561 tablet_service.cc:1467] Processing CreateTablet for tablet 7d899bc49aaf485d82d7344fb27b6c11 (DEFAULT_TABLE table=test-table [id=2f79fb777a844f2da5d0dd02c3c0af57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:58.669890 7958 tablet_service.cc:1467] Processing CreateTablet for tablet 7d899bc49aaf485d82d7344fb27b6c11 (DEFAULT_TABLE table=test-table [id=2f79fb777a844f2da5d0dd02c3c0af57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:58.669924 7561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d899bc49aaf485d82d7344fb27b6c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.670004 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d899bc49aaf485d82d7344fb27b6c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.670687 8115 ts_tablet_manager.cc:1918] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:58.670768 8115 log.cc:1196] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/49212783f8e448d980c3e6d7735f992f
I20260807 22:35:58.671073 8115 ts_tablet_manager.cc:1939] T 49212783f8e448d980c3e6d7735f992f P 316d25c0c023480fb16339ad4e7b54cb: Deleting consensus metadata
I20260807 22:35:58.671468 8081 tablet_bootstrap.cc:492] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: Bootstrap starting.
I20260807 22:35:58.671538 7345 catalog_manager.cc:4841] TS 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181): tablet 49212783f8e448d980c3e6d7735f992f (table test-table [id=833574054697457c9805c2a77a6d9824]) successfully deleted
I20260807 22:35:58.671576 8082 tablet_bootstrap.cc:492] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: Bootstrap starting.
I20260807 22:35:58.671772 7754 tablet_service.cc:1467] Processing CreateTablet for tablet 7d899bc49aaf485d82d7344fb27b6c11 (DEFAULT_TABLE table=test-table [id=2f79fb777a844f2da5d0dd02c3c0af57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:58.671870 7754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d899bc49aaf485d82d7344fb27b6c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.671906 8081 tablet_bootstrap.cc:654] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.672011 8082 tablet_bootstrap.cc:654] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.672667 8082 tablet_bootstrap.cc:492] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: No bootstrap required, opened a new log
I20260807 22:35:58.672742 8082 ts_tablet_manager.cc:1397] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:58.672897 8082 raft_consensus.cc:348] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.672976 8082 raft_consensus.cc:374] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.672992 8083 tablet_bootstrap.cc:492] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: Bootstrap starting.
I20260807 22:35:58.673009 8082 raft_consensus.cc:729] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbb4315bd6c411e867600e8a62f62ac, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.673067 8082 consensus_queue.cc:260] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.673183 8082 ts_tablet_manager.cc:1428] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:58.673398 8083 tablet_bootstrap.cc:654] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.674443 8116 ts_tablet_manager.cc:1905] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:58.674504 8117 ts_tablet_manager.cc:1918] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:58.674628 8117 log.cc:1196] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/49212783f8e448d980c3e6d7735f992f
I20260807 22:35:58.674901 8117 ts_tablet_manager.cc:1939] T 49212783f8e448d980c3e6d7735f992f P 5bbb4315bd6c411e867600e8a62f62ac: Deleting consensus metadata
I20260807 22:35:58.675321 7343 catalog_manager.cc:4841] TS 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933): tablet 49212783f8e448d980c3e6d7735f992f (table test-table [id=833574054697457c9805c2a77a6d9824]) successfully deleted
I20260807 22:35:58.676103 8116 ts_tablet_manager.cc:1918] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:58.676234 8116 log.cc:1196] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/49212783f8e448d980c3e6d7735f992f
I20260807 22:35:58.676522 8116 ts_tablet_manager.cc:1939] T 49212783f8e448d980c3e6d7735f992f P 4080b3e078b5401aa3c45a8dceb41f45: Deleting consensus metadata
I20260807 22:35:58.676947 7347 catalog_manager.cc:4841] TS 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193:42363): tablet 49212783f8e448d980c3e6d7735f992f (table test-table [id=833574054697457c9805c2a77a6d9824]) successfully deleted
I20260807 22:35:58.677227 8083 tablet_bootstrap.cc:492] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: No bootstrap required, opened a new log
I20260807 22:35:58.677284 8083 ts_tablet_manager.cc:1397] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 22:35:58.677331 8081 tablet_bootstrap.cc:492] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: No bootstrap required, opened a new log
I20260807 22:35:58.677379 8081 ts_tablet_manager.cc:1397] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 22:35:58.677456 8083 raft_consensus.cc:348] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.677527 8083 raft_consensus.cc:374] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.677505 8081 raft_consensus.cc:348] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.677547 8083 raft_consensus.cc:729] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 316d25c0c023480fb16339ad4e7b54cb, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.677567 8081 raft_consensus.cc:374] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.677589 8081 raft_consensus.cc:729] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.677632 8081 consensus_queue.cc:260] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.677598 8083 consensus_queue.cc:260] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.677747 8083 ts_tablet_manager.cc:1428] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.677977 8081 ts_tablet_manager.cc:1428] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:58.739454 8094 raft_consensus.cc:482] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:58.739588 8094 raft_consensus.cc:504] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.739972 8094 leader_election.cc:288] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:58.744237 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.740293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:34747 (local address 127.28.82.195:42933)
0807 22:35:58.740344 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.740347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.740466 (+ 119us) server_negotiation.cc:407] Connection header received
0807 22:35:58.740597 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.740599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.740612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.740668 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:58.740963 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.741722 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.743342 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.743883 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.743970 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.743980 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.744124 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.744164 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 22:35:58.744258 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.740187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:42933 (local address 127.28.82.193:34747)
0807 22:35:58.740388 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.740400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.740510 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.740727 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.740728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.740731 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:58.740815 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.740817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.741836 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.741837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.743234 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.743237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.743974 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.743975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.744037 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.744041 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.744116 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.744165 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":38,"thread_start_us":64,"threads_started":1}
I20260807 22:35:58.744557 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d899bc49aaf485d82d7344fb27b6c11" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbb4315bd6c411e867600e8a62f62ac" is_pre_election: true
I20260807 22:35:58.744685 7984 raft_consensus.cc:2454] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 0.
I20260807 22:35:58.744905 7491 leader_election.cc:302] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4080b3e078b5401aa3c45a8dceb41f45, 5bbb4315bd6c411e867600e8a62f62ac; no voters:
I20260807 22:35:58.744983 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.740247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:53567 (local address 127.28.82.194:46181)
0807 22:35:58.740319 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.740322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.740592 (+ 270us) server_negotiation.cc:407] Connection header received
0807 22:35:58.740778 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.740779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.740789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.740880 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:35:58.741049 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.741797 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.744017 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.744545 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.744623 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.744631 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.744872 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.744921 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:35:58.745023 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.740187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:46181 (local address 127.28.82.193:53567)
0807 22:35:58.740494 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.740506 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.740683 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.740887 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.740888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.740889 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:35:58.740964 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.740965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.741881 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.741882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.743234 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.743237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.744693 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.744695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.744765 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.744769 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.744850 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.744891 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":186,"thread_start_us":107,"threads_started":1}
I20260807 22:35:58.745047 8094 raft_consensus.cc:2781] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:58.745090 8094 raft_consensus.cc:482] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:58.745113 8094 raft_consensus.cc:3037] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.745335 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d899bc49aaf485d82d7344fb27b6c11" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb" is_pre_election: true
I20260807 22:35:58.745414 7779 raft_consensus.cc:2454] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 0.
I20260807 22:35:58.746057 8094 raft_consensus.cc:504] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.746203 8094 leader_election.cc:288] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 election: Requested vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:58.746351 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d899bc49aaf485d82d7344fb27b6c11" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb"
I20260807 22:35:58.746430 7779 raft_consensus.cc:3037] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.746407 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d899bc49aaf485d82d7344fb27b6c11" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbb4315bd6c411e867600e8a62f62ac"
I20260807 22:35:58.746507 7984 raft_consensus.cc:3037] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.747092 7779 raft_consensus.cc:2454] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 1.
I20260807 22:35:58.747300 7492 leader_election.cc:302] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 316d25c0c023480fb16339ad4e7b54cb, 4080b3e078b5401aa3c45a8dceb41f45; no voters:
I20260807 22:35:58.747400 7984 raft_consensus.cc:2454] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 1.
I20260807 22:35:58.747416 8094 raft_consensus.cc:2781] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:58.747550 8094 raft_consensus.cc:686] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 LEADER]: Becoming Leader. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Running, Role: LEADER
I20260807 22:35:58.747623 8094 consensus_queue.cc:237] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.748495 7362 catalog_manager.cc:5495] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } health_report { overall_health: UNKNOWN } } }
Aug 07 22:35:58 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142157, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.254@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:58.798786 8131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.791227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:44643 (local address 127.0.0.1:54082)
0807 22:35:58.791416 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:58.791434 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:58.791494 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:58.791642 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:58.791642 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:58.791967 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:35:58.792018 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.792020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.792766 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.792767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.793290 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:58.793292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.793648 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.793649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:58.793705 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.794256 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:35:58.794261 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:35:58.795655 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:35:58.797578 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.797586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.797588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.797794 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.797992 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:35:58.797993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:35:58.797993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:35:58.798056 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:35:58.798359 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:35:58.798367 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:35:58.798415 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 22:35:58.798557 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 22:35:58.798698 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":115,"threads_started":1}
I20260807 22:35:58.798882 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:58.791299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.28.82.254:44643)
0807 22:35:58.791386 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:58.791388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:58.791498 (+ 110us) server_negotiation.cc:407] Connection header received
0807 22:35:58.791544 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:58.791545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:58.791555 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:58.791621 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:58.792120 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.792684 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.793374 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:58.793575 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:58.793640 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:58.795752 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:58.795754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:58.795754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:58.795761 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:58.797410 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.797870 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.797871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.797871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.797929 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:58.798143 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:58.798144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:58.798145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:58.798264 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:58.798332 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:58.798600 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 22:35:58.798755 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":57}
I20260807 22:35:58.800843 7362 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:54082:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 22:35:58.801054 7362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:58.802690 7754 tablet_service.cc:1467] Processing CreateTablet for tablet a71d4099968d4ac5b146b232d41ab237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c334c17d9d41434d809c04428c662b0b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:58.802711 7958 tablet_service.cc:1467] Processing CreateTablet for tablet a71d4099968d4ac5b146b232d41ab237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c334c17d9d41434d809c04428c662b0b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:58.802728 7561 tablet_service.cc:1467] Processing CreateTablet for tablet a71d4099968d4ac5b146b232d41ab237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c334c17d9d41434d809c04428c662b0b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 22:35:58.802870 7754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71d4099968d4ac5b146b232d41ab237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.802872 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71d4099968d4ac5b146b232d41ab237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.802872 7561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71d4099968d4ac5b146b232d41ab237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:58.804106 8083 tablet_bootstrap.cc:492] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: Bootstrap starting.
I20260807 22:35:58.804390 8082 tablet_bootstrap.cc:492] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: Bootstrap starting.
I20260807 22:35:58.804397 8081 tablet_bootstrap.cc:492] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: Bootstrap starting.
I20260807 22:35:58.804546 8083 tablet_bootstrap.cc:654] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.804754 8082 tablet_bootstrap.cc:654] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.804834 8081 tablet_bootstrap.cc:654] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:58.805305 8083 tablet_bootstrap.cc:492] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: No bootstrap required, opened a new log
I20260807 22:35:58.805364 8083 ts_tablet_manager.cc:1397] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:58.805502 8081 tablet_bootstrap.cc:492] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: No bootstrap required, opened a new log
I20260807 22:35:58.805568 8081 ts_tablet_manager.cc:1397] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:58.805524 8083 raft_consensus.cc:348] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.805609 8083 raft_consensus.cc:374] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.805642 8083 raft_consensus.cc:729] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 316d25c0c023480fb16339ad4e7b54cb, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.805687 8083 consensus_queue.cc:260] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.805722 8081 raft_consensus.cc:348] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.805799 8081 raft_consensus.cc:374] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.805821 8081 raft_consensus.cc:729] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.805819 8083 ts_tablet_manager.cc:1428] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:58.805863 8081 consensus_queue.cc:260] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.805858 8090 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:58.805953 8081 ts_tablet_manager.cc:1428] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.805955 8090 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:58.805995 8094 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:58.806073 8094 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:58.806680 8082 tablet_bootstrap.cc:492] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: No bootstrap required, opened a new log
I20260807 22:35:58.806737 8082 ts_tablet_manager.cc:1397] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 22:35:58.806838 8082 raft_consensus.cc:348] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.806888 8082 raft_consensus.cc:374] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:58.806902 8082 raft_consensus.cc:729] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbb4315bd6c411e867600e8a62f62ac, State: Initialized, Role: FOLLOWER
I20260807 22:35:58.806959 8082 consensus_queue.cc:260] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.807065 8082 ts_tablet_manager.cc:1428] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:58.807103 8105 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:58.807210 8105 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:58.991698 8094 raft_consensus.cc:482] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:58.991811 8094 raft_consensus.cc:504] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.991978 8094 leader_election.cc:288] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:58.992218 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71d4099968d4ac5b146b232d41ab237" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbb4315bd6c411e867600e8a62f62ac" is_pre_election: true
I20260807 22:35:58.992336 7984 raft_consensus.cc:2454] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 0.
I20260807 22:35:58.992338 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71d4099968d4ac5b146b232d41ab237" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb" is_pre_election: true
I20260807 22:35:58.992496 7779 raft_consensus.cc:2454] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 0.
I20260807 22:35:58.992512 7491 leader_election.cc:302] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4080b3e078b5401aa3c45a8dceb41f45, 5bbb4315bd6c411e867600e8a62f62ac; no voters:
I20260807 22:35:58.992619 8094 raft_consensus.cc:2781] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:58.992682 8094 raft_consensus.cc:482] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:58.992707 8094 raft_consensus.cc:3037] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.993392 8094 raft_consensus.cc:504] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.993536 8094 leader_election.cc:288] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 election: Requested vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:58.993676 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71d4099968d4ac5b146b232d41ab237" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb"
I20260807 22:35:58.993729 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71d4099968d4ac5b146b232d41ab237" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbb4315bd6c411e867600e8a62f62ac"
I20260807 22:35:58.993768 7779 raft_consensus.cc:3037] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.993810 7984 raft_consensus.cc:3037] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:58.994381 7779 raft_consensus.cc:2454] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 1.
I20260807 22:35:58.994549 7984 raft_consensus.cc:2454] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 1.
I20260807 22:35:58.994637 7492 leader_election.cc:302] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 316d25c0c023480fb16339ad4e7b54cb, 4080b3e078b5401aa3c45a8dceb41f45; no voters:
I20260807 22:35:58.994750 8094 raft_consensus.cc:2781] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:58.994807 8094 raft_consensus.cc:686] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 LEADER]: Becoming Leader. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Running, Role: LEADER
I20260807 22:35:58.994844 8094 consensus_queue.cc:237] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:58.995190 8124 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: TxnStatusTablet state changed. Reason: New leader 4080b3e078b5401aa3c45a8dceb41f45. Latest consensus state: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:58.995288 8124 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:58.995486 8136 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 22:35:58.995618 8136 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 22:35:58.995577 7362 catalog_manager.cc:5495] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } health_report { overall_health: UNKNOWN } } }
I20260807 22:35:59.035249 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:59.032682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.28.82.193:42363)
0807 22:35:59.032748 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:59.032750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:59.032760 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:35:59.032797 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:59.032798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:59.032817 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:59.032891 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:59.033033 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.033550 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.034341 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.034605 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.034688 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:59.034879 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:59.034996 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:59.035078 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:59.035136 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 22:35:59.035164 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:35:59.035252 8131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:59.032425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42363 (local address 127.0.0.1:40306)
0807 22:35:59.032507 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:59.032520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:59.032578 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:59.032901 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:59.032902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:59.032903 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:59.032959 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:59.032962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.033639 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.033640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:59.034247 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:59.034249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.034687 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.034688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:59.034761 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:59.034784 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:59.035062 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:59.035074 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 22:35:59.035130 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 22:35:59.035183 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 22:35:59.036451 7779 raft_consensus.cc:1261] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Refusing update from remote peer 4080b3e078b5401aa3c45a8dceb41f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:59.036514 7984 raft_consensus.cc:1261] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Refusing update from remote peer 4080b3e078b5401aa3c45a8dceb41f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:59.036808 8094 consensus_queue.cc:1035] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:59.036962 8124 consensus_queue.cc:1035] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:59.037465 8090 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: TxnStatusTablet state changed. Reason: New leader 4080b3e078b5401aa3c45a8dceb41f45. Latest consensus state: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:59.037570 8090 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:59.037662 8105 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: TxnStatusTablet state changed. Reason: New leader 4080b3e078b5401aa3c45a8dceb41f45. Latest consensus state: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:59.037756 8105 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:59.038050 8124 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:59.038132 8124 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:59.038636 8090 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:59.038727 8090 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 316d25c0c023480fb16339ad4e7b54cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:59.039115 8094 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:59.039220 8094 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 4080b3e078b5401aa3c45a8dceb41f45: This TxnStatusTablet replica's current role is: LEADER
I20260807 22:35:59.039426 8105 tablet_replica.cc:439] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } } }
I20260807 22:35:59.039505 8105 tablet_replica.cc:442] T a71d4099968d4ac5b146b232d41ab237 P 5bbb4315bd6c411e867600e8a62f62ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 22:35:59.043329 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:59.040841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.28.82.193:42363)
0807 22:35:59.040920 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:59.040922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:59.040940 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:35:59.041025 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:59.041026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:59.041034 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:59.041103 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:59.041242 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.041774 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.042639 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.042824 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.042912 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:59.043063 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:59.043100 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:59.043171 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:59.043234 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 22:35:59.043254 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:35:59.048534 8152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:59.045787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42363 (local address 127.28.82.193:60843)
0807 22:35:59.045972 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:59.045981 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:35:59.046029 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:35:59.046164 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:35:59.046165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:35:59.046166 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 22:35:59.046217 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:59.046218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.047075 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.047076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:59.047635 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:35:59.047636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.048129 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.048130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:35:59.048194 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:59.048209 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 22:35:59.048382 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 22:35:59.048384 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:35:59.048418 (+ 34us) client_negotiation.cc:240] Negotiation successful
0807 22:35:59.048452 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260807 22:35:59.048543 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:59.045866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:60843 (local address 127.28.82.193:42363)
0807 22:35:59.045925 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:59.045927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:59.046036 (+ 109us) server_negotiation.cc:407] Connection header received
0807 22:35:59.046081 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:59.046082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:59.046092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:59.046143 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:59.046275 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.046996 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.047747 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.048042 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.048121 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:59.048291 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:59.048322 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:59.048376 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:59.048421 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 22:35:59.048440 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:35:59.050854 7779 raft_consensus.cc:1261] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Refusing update from remote peer 4080b3e078b5401aa3c45a8dceb41f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:59.050853 7984 raft_consensus.cc:1261] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Refusing update from remote peer 4080b3e078b5401aa3c45a8dceb41f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:59.051293 8147 consensus_queue.cc:1035] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:59.051427 8094 consensus_queue.cc:1035] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:59.063966 7362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:59.064100 7362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:59.064749 7362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: Sending DeleteTablet for 3 replicas of tablet 7d899bc49aaf485d82d7344fb27b6c11
I20260807 22:35:59.065053 7754 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d899bc49aaf485d82d7344fb27b6c11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:35620
I20260807 22:35:59.065127 7561 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d899bc49aaf485d82d7344fb27b6c11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40304
I20260807 22:35:59.065162 7958 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d899bc49aaf485d82d7344fb27b6c11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:46122
I20260807 22:35:59.065258 8117 tablet_replica.cc:330] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: stopping tablet replica
I20260807 22:35:59.065325 8115 tablet_replica.cc:330] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: stopping tablet replica
I20260807 22:35:59.065330 8117 raft_consensus.cc:2229] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:59.065382 8117 raft_consensus.cc:2258] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.065385 8115 raft_consensus.cc:2229] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:59.065421 8115 raft_consensus.cc:2258] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.065483 8116 tablet_replica.cc:330] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: stopping tablet replica
I20260807 22:35:59.065536 8116 raft_consensus.cc:2229] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:59.065619 8116 raft_consensus.cc:2258] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.065728 8117 ts_tablet_manager.cc:1905] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.066131 8116 ts_tablet_manager.cc:1905] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.067339 8117 ts_tablet_manager.cc:1918] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:59.067387 8117 log.cc:1196] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/7d899bc49aaf485d82d7344fb27b6c11
I20260807 22:35:59.067572 8117 ts_tablet_manager.cc:1939] T 7d899bc49aaf485d82d7344fb27b6c11 P 5bbb4315bd6c411e867600e8a62f62ac: Deleting consensus metadata
I20260807 22:35:59.067818 8116 ts_tablet_manager.cc:1918] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:59.067873 8116 log.cc:1196] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/7d899bc49aaf485d82d7344fb27b6c11
I20260807 22:35:59.067863 7343 catalog_manager.cc:4841] TS 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933): tablet 7d899bc49aaf485d82d7344fb27b6c11 (table test-table [id=2f79fb777a844f2da5d0dd02c3c0af57]) successfully deleted
I20260807 22:35:59.068122 8116 ts_tablet_manager.cc:1939] T 7d899bc49aaf485d82d7344fb27b6c11 P 4080b3e078b5401aa3c45a8dceb41f45: Deleting consensus metadata
I20260807 22:35:59.068518 7347 catalog_manager.cc:4841] TS 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193:42363): tablet 7d899bc49aaf485d82d7344fb27b6c11 (table test-table [id=2f79fb777a844f2da5d0dd02c3c0af57]) successfully deleted
I20260807 22:35:59.072153 8115 ts_tablet_manager.cc:1905] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.074255 8115 ts_tablet_manager.cc:1918] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:59.074335 8115 log.cc:1196] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/7d899bc49aaf485d82d7344fb27b6c11
I20260807 22:35:59.074627 8115 ts_tablet_manager.cc:1939] T 7d899bc49aaf485d82d7344fb27b6c11 P 316d25c0c023480fb16339ad4e7b54cb: Deleting consensus metadata
I20260807 22:35:59.075008 7345 catalog_manager.cc:4841] TS 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181): tablet 7d899bc49aaf485d82d7344fb27b6c11 (table test-table [id=2f79fb777a844f2da5d0dd02c3c0af57]) successfully deleted
Aug 07 22:35:59 dist-test-slave-7kzw krb5kdc[7298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 22:35:59 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:35:59 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:35:59.096148 7435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:35:59.089950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.28.82.254:44643)
0807 22:35:59.090005 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:59.090007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:59.090114 (+ 107us) server_negotiation.cc:407] Connection header received
0807 22:35:59.090152 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:59.090153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:59.090167 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:59.090238 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:59.090903 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.091722 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.092463 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.092771 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.092853 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:59.096080 (+ 3227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54084: BlockingRecv error: recv got EOF from 127.0.0.1:54084 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 07 22:35:59 dist-test-slave-7kzw krb5kdc[7298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM
I20260807 22:35:59.104342 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:59.096999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54086 (local address 127.28.82.254:44643)
0807 22:35:59.097043 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:59.097045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:59.097136 (+ 91us) server_negotiation.cc:407] Connection header received
0807 22:35:59.097242 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:59.097243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:59.097254 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:59.097306 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:35:59.097765 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.098217 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.098988 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.099172 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.099244 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:59.101651 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:35:59.101652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:35:59.101653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:35:59.101660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:35:59.103199 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:59.103562 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:59.103563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:59.103563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:59.103628 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:35:59.103832 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:35:59.103833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:35:59.103834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:35:59.103944 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:35:59.104050 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:59.104151 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 22:35:59.104237 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:35:59.105924 7362 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:59.106114 7362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:59.107570 7754 tablet_service.cc:1467] Processing CreateTablet for tablet cdb2801f957b4682877498afc930f24b (DEFAULT_TABLE table=test-table [id=57f0d629d85e424dabfe5e5fc4390ae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:59.107614 7561 tablet_service.cc:1467] Processing CreateTablet for tablet cdb2801f957b4682877498afc930f24b (DEFAULT_TABLE table=test-table [id=57f0d629d85e424dabfe5e5fc4390ae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:59.107720 7754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb2801f957b4682877498afc930f24b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:59.107774 7561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb2801f957b4682877498afc930f24b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:59.108829 8081 tablet_bootstrap.cc:492] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: Bootstrap starting.
I20260807 22:35:59.109089 8083 tablet_bootstrap.cc:492] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: Bootstrap starting.
I20260807 22:35:59.109128 7958 tablet_service.cc:1467] Processing CreateTablet for tablet cdb2801f957b4682877498afc930f24b (DEFAULT_TABLE table=test-table [id=57f0d629d85e424dabfe5e5fc4390ae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:59.109256 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb2801f957b4682877498afc930f24b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:59.109316 8081 tablet_bootstrap.cc:654] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:59.109529 8083 tablet_bootstrap.cc:654] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:59.110201 8083 tablet_bootstrap.cc:492] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: No bootstrap required, opened a new log
I20260807 22:35:59.110267 8083 ts_tablet_manager.cc:1397] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:59.110438 8083 raft_consensus.cc:348] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.110597 8083 raft_consensus.cc:374] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:59.110605 8082 tablet_bootstrap.cc:492] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: Bootstrap starting.
I20260807 22:35:59.110626 8083 raft_consensus.cc:729] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 316d25c0c023480fb16339ad4e7b54cb, State: Initialized, Role: FOLLOWER
I20260807 22:35:59.110672 8083 consensus_queue.cc:260] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.110780 8083 ts_tablet_manager.cc:1428] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 22:35:59.110924 8082 tablet_bootstrap.cc:654] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:59.111289 8081 tablet_bootstrap.cc:492] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: No bootstrap required, opened a new log
I20260807 22:35:59.111344 8081 ts_tablet_manager.cc:1397] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 22:35:59.111505 8081 raft_consensus.cc:348] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.111578 8081 raft_consensus.cc:374] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:59.111599 8081 raft_consensus.cc:729] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Initialized, Role: FOLLOWER
I20260807 22:35:59.111594 8082 tablet_bootstrap.cc:492] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: No bootstrap required, opened a new log
I20260807 22:35:59.111634 8082 ts_tablet_manager.cc:1397] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:59.111644 8081 consensus_queue.cc:260] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.111737 8081 ts_tablet_manager.cc:1428] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:59.111755 8082 raft_consensus.cc:348] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.111820 8082 raft_consensus.cc:374] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:59.111855 8082 raft_consensus.cc:729] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbb4315bd6c411e867600e8a62f62ac, State: Initialized, Role: FOLLOWER
I20260807 22:35:59.111891 8082 consensus_queue.cc:260] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.111987 8082 ts_tablet_manager.cc:1428] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:59.242956 8124 raft_consensus.cc:482] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:59.243067 8124 raft_consensus.cc:504] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.243245 8124 leader_election.cc:288] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:59.243490 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdb2801f957b4682877498afc930f24b" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb" is_pre_election: true
I20260807 22:35:59.243588 7779 raft_consensus.cc:2454] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 0.
I20260807 22:35:59.243566 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdb2801f957b4682877498afc930f24b" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbb4315bd6c411e867600e8a62f62ac" is_pre_election: true
I20260807 22:35:59.243741 7984 raft_consensus.cc:2454] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 0.
I20260807 22:35:59.243773 7492 leader_election.cc:302] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 316d25c0c023480fb16339ad4e7b54cb, 4080b3e078b5401aa3c45a8dceb41f45; no voters:
I20260807 22:35:59.243870 8124 raft_consensus.cc:2781] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:59.243904 8124 raft_consensus.cc:482] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:59.243923 8124 raft_consensus.cc:3037] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:59.244825 8124 raft_consensus.cc:504] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.244968 8124 leader_election.cc:288] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 election: Requested vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:59.245193 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdb2801f957b4682877498afc930f24b" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb"
I20260807 22:35:59.245282 7779 raft_consensus.cc:3037] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:59.245306 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdb2801f957b4682877498afc930f24b" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbb4315bd6c411e867600e8a62f62ac"
I20260807 22:35:59.245391 7984 raft_consensus.cc:3037] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:59.246008 7779 raft_consensus.cc:2454] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 1.
I20260807 22:35:59.246021 7984 raft_consensus.cc:2454] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 1.
I20260807 22:35:59.246234 7491 leader_election.cc:302] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4080b3e078b5401aa3c45a8dceb41f45, 5bbb4315bd6c411e867600e8a62f62ac; no voters:
I20260807 22:35:59.246337 8124 raft_consensus.cc:2781] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:59.246415 8124 raft_consensus.cc:686] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 LEADER]: Becoming Leader. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Running, Role: LEADER
I20260807 22:35:59.246505 8124 consensus_queue.cc:237] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.247202 7363 catalog_manager.cc:5495] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } health_report { overall_health: UNKNOWN } } }
I20260807 22:35:59.260455 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:59.258180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.28.82.193:42363)
0807 22:35:59.258240 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 22:35:59.258243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:35:59.258256 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:35:59.258277 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:35:59.258278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:35:59.258286 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:35:59.258334 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:35:59.258519 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.258983 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.259766 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:35:59.259951 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:35:59.260015 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:35:59.260215 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:35:59.260247 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:35:59.260303 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 22:35:59.260384 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 22:35:59.260401 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:35:59.261428 7779 raft_consensus.cc:1261] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Refusing update from remote peer 4080b3e078b5401aa3c45a8dceb41f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:59.261524 7984 raft_consensus.cc:1261] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Refusing update from remote peer 4080b3e078b5401aa3c45a8dceb41f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:59.261754 8124 consensus_queue.cc:1035] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:59.261873 8094 consensus_queue.cc:1035] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:59.264510 7363 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:59.264609 7363 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:59.265326 7363 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: Sending DeleteTablet for 3 replicas of tablet cdb2801f957b4682877498afc930f24b
I20260807 22:35:59.265605 7754 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb2801f957b4682877498afc930f24b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:35620
I20260807 22:35:59.265601 7561 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb2801f957b4682877498afc930f24b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40304
I20260807 22:35:59.265726 7958 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb2801f957b4682877498afc930f24b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:46122
I20260807 22:35:59.265751 8116 tablet_replica.cc:330] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: stopping tablet replica
I20260807 22:35:59.265815 8116 raft_consensus.cc:2229] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:59.265825 8115 tablet_replica.cc:330] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: stopping tablet replica
I20260807 22:35:59.265844 8117 tablet_replica.cc:330] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: stopping tablet replica
I20260807 22:35:59.265879 8116 raft_consensus.cc:2258] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.265880 8115 raft_consensus.cc:2229] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:59.265902 8117 raft_consensus.cc:2229] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:59.265923 8115 raft_consensus.cc:2258] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.265949 8117 raft_consensus.cc:2258] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.266211 8116 ts_tablet_manager.cc:1905] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.266389 7363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:35:59.266572 7363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:35:59.266553 8115 ts_tablet_manager.cc:1905] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.267815 7561 tablet_service.cc:1467] Processing CreateTablet for tablet 25ed97817a8c45fda43df9474537a641 (DEFAULT_TABLE table=test-table [id=d5059bec103e4796a40bf6f85be4f0e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:59.267894 7958 tablet_service.cc:1467] Processing CreateTablet for tablet 25ed97817a8c45fda43df9474537a641 (DEFAULT_TABLE table=test-table [id=d5059bec103e4796a40bf6f85be4f0e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:59.268019 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ed97817a8c45fda43df9474537a641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:59.268023 7561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ed97817a8c45fda43df9474537a641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:59.268201 8115 ts_tablet_manager.cc:1918] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:59.268255 8115 log.cc:1196] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/cdb2801f957b4682877498afc930f24b
I20260807 22:35:59.268488 8115 ts_tablet_manager.cc:1939] T cdb2801f957b4682877498afc930f24b P 316d25c0c023480fb16339ad4e7b54cb: Deleting consensus metadata
I20260807 22:35:59.268853 8117 ts_tablet_manager.cc:1905] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.269321 8081 tablet_bootstrap.cc:492] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45: Bootstrap starting.
I20260807 22:35:59.268021 8116 ts_tablet_manager.cc:1918] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:59.269569 8082 tablet_bootstrap.cc:492] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: Bootstrap starting.
I20260807 22:35:59.269580 8116 log.cc:1196] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/cdb2801f957b4682877498afc930f24b
I20260807 22:35:59.269711 8081 tablet_bootstrap.cc:654] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:59.269855 8116 ts_tablet_manager.cc:1939] T cdb2801f957b4682877498afc930f24b P 4080b3e078b5401aa3c45a8dceb41f45: Deleting consensus metadata
I20260807 22:35:59.269912 8082 tablet_bootstrap.cc:654] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:59.270205 7347 catalog_manager.cc:4841] TS 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193:42363): tablet cdb2801f957b4682877498afc930f24b (table test-table [id=57f0d629d85e424dabfe5e5fc4390ae1]) successfully deleted
I20260807 22:35:59.270328 8117 ts_tablet_manager.cc:1918] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:35:59.270392 8117 log.cc:1196] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/cdb2801f957b4682877498afc930f24b
I20260807 22:35:59.270581 8082 tablet_bootstrap.cc:492] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: No bootstrap required, opened a new log
I20260807 22:35:59.270629 7345 catalog_manager.cc:4841] TS 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181): tablet cdb2801f957b4682877498afc930f24b (table test-table [id=57f0d629d85e424dabfe5e5fc4390ae1]) successfully deleted
I20260807 22:35:59.270641 8082 ts_tablet_manager.cc:1397] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 22:35:59.270694 8117 ts_tablet_manager.cc:1939] T cdb2801f957b4682877498afc930f24b P 5bbb4315bd6c411e867600e8a62f62ac: Deleting consensus metadata
I20260807 22:35:59.270695 7754 tablet_service.cc:1467] Processing CreateTablet for tablet 25ed97817a8c45fda43df9474537a641 (DEFAULT_TABLE table=test-table [id=d5059bec103e4796a40bf6f85be4f0e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:35:59.270808 7754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ed97817a8c45fda43df9474537a641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:35:59.270759 8082 raft_consensus.cc:348] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.270845 8082 raft_consensus.cc:374] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:59.270881 8082 raft_consensus.cc:729] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbb4315bd6c411e867600e8a62f62ac, State: Initialized, Role: FOLLOWER
I20260807 22:35:59.270934 8082 consensus_queue.cc:260] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.271054 8082 ts_tablet_manager.cc:1428] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 22:35:59.271377 8081 tablet_bootstrap.cc:492] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45: No bootstrap required, opened a new log
I20260807 22:35:59.271425 8081 ts_tablet_manager.cc:1397] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:35:59.271514 8081 raft_consensus.cc:348] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.271566 8081 raft_consensus.cc:374] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:59.271581 8081 raft_consensus.cc:729] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Initialized, Role: FOLLOWER
I20260807 22:35:59.271651 8081 consensus_queue.cc:260] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.271759 8081 ts_tablet_manager.cc:1428] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:59.272037 8083 tablet_bootstrap.cc:492] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: Bootstrap starting.
I20260807 22:35:59.272245 7343 catalog_manager.cc:4841] TS 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933): tablet cdb2801f957b4682877498afc930f24b (table test-table [id=57f0d629d85e424dabfe5e5fc4390ae1]) successfully deleted
I20260807 22:35:59.272451 8083 tablet_bootstrap.cc:654] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: Neither blocks nor log segments found. Creating new log.
I20260807 22:35:59.273173 8083 tablet_bootstrap.cc:492] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: No bootstrap required, opened a new log
I20260807 22:35:59.273218 8083 ts_tablet_manager.cc:1397] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:35:59.273322 8083 raft_consensus.cc:348] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.273370 8083 raft_consensus.cc:374] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:35:59.273384 8083 raft_consensus.cc:729] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 316d25c0c023480fb16339ad4e7b54cb, State: Initialized, Role: FOLLOWER
I20260807 22:35:59.273448 8083 consensus_queue.cc:260] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.273546 8083 ts_tablet_manager.cc:1428] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:35:59.368197 8094 raft_consensus.cc:482] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:35:59.368302 8094 raft_consensus.cc:504] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.368487 8094 leader_election.cc:288] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:59.368693 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ed97817a8c45fda43df9474537a641" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb" is_pre_election: true
I20260807 22:35:59.368809 7779 raft_consensus.cc:2454] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 0.
I20260807 22:35:59.368777 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ed97817a8c45fda43df9474537a641" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbb4315bd6c411e867600e8a62f62ac" is_pre_election: true
I20260807 22:35:59.368907 7984 raft_consensus.cc:2454] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 0.
I20260807 22:35:59.369024 7492 leader_election.cc:302] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 316d25c0c023480fb16339ad4e7b54cb, 4080b3e078b5401aa3c45a8dceb41f45; no voters:
I20260807 22:35:59.369138 8094 raft_consensus.cc:2781] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:35:59.369176 8094 raft_consensus.cc:482] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:35:59.369211 8094 raft_consensus.cc:3037] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:59.370121 8094 raft_consensus.cc:504] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.370285 8094 leader_election.cc:288] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 election: Requested vote from peers 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181), 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933)
I20260807 22:35:59.370501 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ed97817a8c45fda43df9474537a641" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbb4315bd6c411e867600e8a62f62ac"
I20260807 22:35:59.370528 7779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ed97817a8c45fda43df9474537a641" candidate_uuid: "4080b3e078b5401aa3c45a8dceb41f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "316d25c0c023480fb16339ad4e7b54cb"
I20260807 22:35:59.370635 7779 raft_consensus.cc:3037] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:59.370635 7984 raft_consensus.cc:3037] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:35:59.371276 7779 raft_consensus.cc:2454] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 1.
I20260807 22:35:59.371289 7984 raft_consensus.cc:2454] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4080b3e078b5401aa3c45a8dceb41f45 in term 1.
I20260807 22:35:59.371496 7491 leader_election.cc:302] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4080b3e078b5401aa3c45a8dceb41f45, 5bbb4315bd6c411e867600e8a62f62ac; no voters:
I20260807 22:35:59.371595 8094 raft_consensus.cc:2781] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:35:59.371655 8094 raft_consensus.cc:686] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 LEADER]: Becoming Leader. State: Replica: 4080b3e078b5401aa3c45a8dceb41f45, State: Running, Role: LEADER
I20260807 22:35:59.371721 8094 consensus_queue.cc:237] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } }
I20260807 22:35:59.372334 7363 catalog_manager.cc:5495] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "4080b3e078b5401aa3c45a8dceb41f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4080b3e078b5401aa3c45a8dceb41f45" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 } health_report { overall_health: UNKNOWN } } }
I20260807 22:35:59.391042 7984 raft_consensus.cc:1261] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Refusing update from remote peer 4080b3e078b5401aa3c45a8dceb41f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:59.391119 7779 raft_consensus.cc:1261] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Refusing update from remote peer 4080b3e078b5401aa3c45a8dceb41f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:35:59.391300 8190 consensus_queue.cc:1035] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbb4315bd6c411e867600e8a62f62ac" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:59.391395 8147 consensus_queue.cc:1035] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "316d25c0c023480fb16339ad4e7b54cb" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:35:59.400880 7362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:59.400969 7362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:35:59.401595 7362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 656d04a541c846db8a8e03d0d1be5478: Sending DeleteTablet for 3 replicas of tablet 25ed97817a8c45fda43df9474537a641
I20260807 22:35:59.401846 7561 tablet_service.cc:1514] Processing DeleteTablet for tablet 25ed97817a8c45fda43df9474537a641 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40304
I20260807 22:35:59.401844 7958 tablet_service.cc:1514] Processing DeleteTablet for tablet 25ed97817a8c45fda43df9474537a641 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:46122
I20260807 22:35:59.401953 8116 tablet_replica.cc:330] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45: stopping tablet replica
I20260807 22:35:59.402024 8116 raft_consensus.cc:2229] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:35:59.402081 8117 tablet_replica.cc:330] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: stopping tablet replica
I20260807 22:35:59.402099 8116 raft_consensus.cc:2258] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.402086 7754 tablet_service.cc:1514] Processing DeleteTablet for tablet 25ed97817a8c45fda43df9474537a641 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:35:59 UTC) from {username='oryx', principal='oryx/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:35620
I20260807 22:35:59.402153 8117 raft_consensus.cc:2229] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:59.402199 8117 raft_consensus.cc:2258] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.402343 8115 tablet_replica.cc:330] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: stopping tablet replica
I20260807 22:35:59.402405 8115 raft_consensus.cc:2229] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:35:59.402426 8116 ts_tablet_manager.cc:1905] T 25ed97817a8c45fda43df9474537a641 P 4080b3e078b5401aa3c45a8dceb41f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.402443 8115 raft_consensus.cc:2258] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:35:59.402438 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 7464
I20260807 22:35:59.405011 8117 ts_tablet_manager.cc:1905] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.406891 8117 ts_tablet_manager.cc:1918] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:59.406950 8117 log.cc:1196] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/25ed97817a8c45fda43df9474537a641
I20260807 22:35:59.407158 8117 ts_tablet_manager.cc:1939] T 25ed97817a8c45fda43df9474537a641 P 5bbb4315bd6c411e867600e8a62f62ac: Deleting consensus metadata
I20260807 22:35:59.407408 8115 ts_tablet_manager.cc:1905] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:35:59.407567 7343 catalog_manager.cc:4841] TS 5bbb4315bd6c411e867600e8a62f62ac (127.28.82.195:42933): tablet 25ed97817a8c45fda43df9474537a641 (table test-table [id=d5059bec103e4796a40bf6f85be4f0e4]) successfully deleted
I20260807 22:35:59.408949 8115 ts_tablet_manager.cc:1918] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 22:35:59.409019 8115 log.cc:1196] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/25ed97817a8c45fda43df9474537a641
I20260807 22:35:59.409250 8115 ts_tablet_manager.cc:1939] T 25ed97817a8c45fda43df9474537a641 P 316d25c0c023480fb16339ad4e7b54cb: Deleting consensus metadata
I20260807 22:35:59.409600 7345 catalog_manager.cc:4841] TS 316d25c0c023480fb16339ad4e7b54cb (127.28.82.194:46181): tablet 25ed97817a8c45fda43df9474537a641 (table test-table [id=d5059bec103e4796a40bf6f85be4f0e4]) successfully deleted
W20260807 22:35:59.419144 7347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:42363 (error 108)
I20260807 22:35:59.419381 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 7662
I20260807 22:35:59.421008 8198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:35:59.419530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42363 (local address 127.0.0.1:40332)
0807 22:35:59.420877 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0807 22:35:59.420939 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:42363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1325,"thread_start_us":40,"threads_started":1}
W20260807 22:35:59.421156 7347 catalog_manager.cc:4584] TS 4080b3e078b5401aa3c45a8dceb41f45 (127.28.82.193:42363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25ed97817a8c45fda43df9474537a641: Network error: Client connection negotiation failed: client connection to 127.28.82.193:42363: connect: Connection refused (error 111)
I20260807 22:35:59.431478 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 7858
I20260807 22:35:59.442574 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 7316
2026-08-07T22:35:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3061 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:35:59 dist-test-slave-7kzw krb5kdc[8201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:35:59 dist-test-slave-7kzw krb5kdc[8201](info): set up 2 sockets
Aug 07 22:35:59 dist-test-slave-7kzw krb5kdc[8201](info): commencing operation
krb5kdc: starting...
W20260807 22:36:00.676090 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:36:00 dist-test-slave-7kzw krb5kdc[8201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:36:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:36:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:36:00.806782 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:40543
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:40543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:00.896178 8219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:00.896432 8219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:00.896476 8219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:00.897635 8219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:36:00.897714 8219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:00.897746 8219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:00.897782 8219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:36:00.897816 8219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:36:00.899456 8219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:40543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:40543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:00.899755 8219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:00.900022 8219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:00.905858 8228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:00.905858 8225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:00.905907 8224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:00.906230 8219 server_base.cc:1034] running on GCE node
I20260807 22:36:00.906689 8219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:00.907032 8219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:00.908200 8219 hybrid_clock.cc:648] HybridClock initialized: now 1786142160908177 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T22:36:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1466 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:36:00 dist-test-slave-7kzw krb5kdc[8236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:36:00 dist-test-slave-7kzw krb5kdc[8236](info): set up 2 sockets
Aug 07 22:36:00 dist-test-slave-7kzw krb5kdc[8236](info): commencing operation
krb5kdc: starting...
W20260807 22:36:02.129858 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:36:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:36:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:36:02.257014 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46879
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:46879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:02.339221 8256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:02.339457 8256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:02.339493 8256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:02.340631 8256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:36:02.340696 8256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:02.340720 8256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:02.340741 8256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:36:02.340775 8256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:36:02.342208 8256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:46879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:02.342496 8256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:02.342726 8256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:02.347715 8266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:02.347779 8264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:02.347710 8263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:02.347965 8256 server_base.cc:1034] running on GCE node
I20260807 22:36:02.348280 8256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:02.348522 8256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:02.349665 8256 hybrid_clock.cc:648] HybridClock initialized: now 1786142162349644 us; error 34 us; skew 500 ppm
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:02.352293 8256 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:36:02.352821 8256 webserver.cc:460] Webserver started at http://127.28.82.254:34231/ using document root <none> and password file <none>
I20260807 22:36:02.353082 8256 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:02.353130 8256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:02.353281 8256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:02.354709 8256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "f4e045978f0246f99aa851aa21d557f5"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "ba1c5d31d0710367254996b39a7fb8d4"
server_key_iv: "1f3ffc258e527938ea656fa9a5e1b65e"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.355114 8256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "f4e045978f0246f99aa851aa21d557f5"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "ba1c5d31d0710367254996b39a7fb8d4"
server_key_iv: "1f3ffc258e527938ea656fa9a5e1b65e"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.356463 8256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:36:02.357239 8274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.357430 8256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:36:02.357538 8256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "f4e045978f0246f99aa851aa21d557f5"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "ba1c5d31d0710367254996b39a7fb8d4"
server_key_iv: "1f3ffc258e527938ea656fa9a5e1b65e"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.357625 8256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:02.371176 8256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:02.373236 8256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:02.373409 8256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:02.377732 8256 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:46879
I20260807 22:36:02.377776 8353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:46879 every 8 connection(s)
I20260807 22:36:02.378115 8256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:36:02.379076 8354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:36:02.381316 8354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Bootstrap starting.
I20260807 22:36:02.381860 8354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Neither blocks nor log segments found. Creating new log.
I20260807 22:36:02.382077 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 8256
I20260807 22:36:02.382212 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:36:02.382249 8354 log.cc:824] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:36:02.382437 29003 external_mini_cluster.cc:1428] Setting key 9036771bfa5b294d0f63bc99b05592fe
I20260807 22:36:02.383111 8354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: No bootstrap required, opened a new log
I20260807 22:36:02.385329 8354 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } }
I20260807 22:36:02.385488 8354 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:36:02.385535 8354 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e045978f0246f99aa851aa21d557f5, State: Initialized, Role: FOLLOWER
I20260807 22:36:02.385692 8354 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } }
I20260807 22:36:02.385800 8354 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:36:02.385847 8354 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:36:02.385900 8354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:36:02.386437 8354 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } }
I20260807 22:36:02.386590 8354 leader_election.cc:302] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4e045978f0246f99aa851aa21d557f5; no voters:
I20260807 22:36:02.386798 8354 leader_election.cc:288] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:36:02.386840 8360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:36:02.387038 8360 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 1 LEADER]: Becoming Leader. State: Replica: f4e045978f0246f99aa851aa21d557f5, State: Running, Role: LEADER
I20260807 22:36:02.387190 8360 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } }
I20260807 22:36:02.387060 8354 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:36:02.387823 8361 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4e045978f0246f99aa851aa21d557f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } } }
I20260807 22:36:02.387918 8361 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:02.388219 8362 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4e045978f0246f99aa851aa21d557f5. Latest consensus state: current_term: 1 leader_uuid: "f4e045978f0246f99aa851aa21d557f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } } }
I20260807 22:36:02.388312 8362 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: This master's current role is: LEADER
W20260807 22:36:02.389411 8385 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:36:02.389469 8385 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:36:02.389515 8372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:36:02.389680 8372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:36:02.391271 8372 catalog_manager.cc:1347] Generated new cluster ID: 2c480cf8963b4ede827bec7bf7880183
I20260807 22:36:02.391328 8372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:36:02.393548 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:02.382972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.28.82.254:46879)
0807 22:36:02.383168 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:02.383178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:02.383212 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:36:02.383656 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:02.383662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:02.383707 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:02.383852 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:02.384487 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.385360 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.386336 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.386997 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.387182 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:02.390125 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:02.390129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:02.390129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:02.390140 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:02.391984 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:02.392365 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:02.392368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:02.392368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:02.392463 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:02.392687 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:02.392688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:02.392688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:02.392926 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:02.393125 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:02.393203 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 22:36:02.393358 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260807 22:36:02.411137 8372 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:36:02.411845 8372 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:36:02.425551 8372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Generated new TSK 0
I20260807 22:36:02.425745 8372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:36:02.455505 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46879
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:02.539077 8394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:02.539281 8394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:02.539304 8394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:02.540455 8394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:02.540495 8394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:02.540521 8394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:36:02.541890 8394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:02.542068 8394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:02.542237 8394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:02.547348 8401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:02.547489 8394 server_base.cc:1034] running on GCE node
W20260807 22:36:02.547335 8403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:02.547338 8400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:02.547812 8394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:02.548015 8394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:02.549154 8394 hybrid_clock.cc:648] HybridClock initialized: now 1786142162549136 us; error 34 us; skew 500 ppm
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:02.551862 8394 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:36:02.552243 8394 webserver.cc:460] Webserver started at http://127.28.82.193:40989/ using document root <none> and password file <none>
I20260807 22:36:02.552416 8394 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:02.552457 8394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:02.552541 8394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:02.553474 8394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "ad8733f3688c4c058553e0e73d148c22"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "0e1c298eac80ed4034b00a630bd8d652"
server_key_iv: "4583f2d9445c203f012e0d8b1c8ec65f"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.553772 8394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "ad8733f3688c4c058553e0e73d148c22"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "0e1c298eac80ed4034b00a630bd8d652"
server_key_iv: "4583f2d9445c203f012e0d8b1c8ec65f"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.555225 8394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:36:02.556039 8415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.556277 8394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:02.556396 8394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "ad8733f3688c4c058553e0e73d148c22"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "0e1c298eac80ed4034b00a630bd8d652"
server_key_iv: "4583f2d9445c203f012e0d8b1c8ec65f"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.556474 8394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:02.586189 8394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:02.588883 8394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:02.589077 8394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:02.589414 8394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:02.589747 8394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:02.589805 8394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.589846 8394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:02.589893 8394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.595938 8394 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:40769
I20260807 22:36:02.595994 8568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:40769 every 8 connection(s)
I20260807 22:36:02.596305 8394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:36:02.601076 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 8394
I20260807 22:36:02.601220 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:36:02.601437 29003 external_mini_cluster.cc:1428] Setting key 243603a486aac76a1e9a204921f2fc78
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:36:02.606065 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:02.597027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:60251 (local address 127.28.82.254:46879)
0807 22:36:02.597119 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:02.597122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:02.597315 (+ 193us) server_negotiation.cc:407] Connection header received
0807 22:36:02.597841 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:02.597843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:02.597855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:02.597914 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:02.598879 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.599402 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.600386 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.600621 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.600701 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:02.603153 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:02.603155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:02.603155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:02.603163 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:02.604604 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:02.605060 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:02.605061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:02.605062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:02.605126 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:02.605393 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:02.605394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:02.605394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:02.605509 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:02.605581 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:02.605873 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 22:36:02.605970 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:36:02.606227 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:02.596989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:60251)
0807 22:36:02.597187 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:02.597225 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:02.597730 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:02.597985 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:02.597989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:02.598587 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:02.598774 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:02.598779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.599519 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.599520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:02.600285 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:02.600287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.600709 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.600710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:02.600830 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:02.601385 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:02.601396 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:02.603054 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:02.604741 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:02.604749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:02.604751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:02.604975 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:02.605198 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:02.605199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:02.605199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:02.605314 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:02.605586 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:02.605594 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:02.605730 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 22:36:02.605835 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:36:02.605988 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260807 22:36:02.606763 8569 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46879
I20260807 22:36:02.606858 8569 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:02.607033 8569 heartbeater.cc:511] Master 127.28.82.254:46879 requested a full tablet report, sending...
I20260807 22:36:02.607576 8301 ts_manager.cc:194] Registered new tserver with Master: ad8733f3688c4c058553e0e73d148c22 (127.28.82.193:40769)
I20260807 22:36:02.608176 8301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:60251
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:36:02.629467 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46879
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:02.711001 8576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:02.711251 8576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:02.711289 8576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:02.712445 8576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:02.712518 8576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:02.712561 8576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:36:02.714023 8576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:02.714308 8576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:02.714570 8576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:02.719755 8583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:02.719739 8581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:02.719743 8585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:02.720193 8576 server_base.cc:1034] running on GCE node
I20260807 22:36:02.720455 8576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:02.720718 8576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:02.721895 8576 hybrid_clock.cc:648] HybridClock initialized: now 1786142162721880 us; error 28 us; skew 500 ppm
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:02.724618 8576 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:36:02.725044 8576 webserver.cc:460] Webserver started at http://127.28.82.194:43311/ using document root <none> and password file <none>
I20260807 22:36:02.725298 8576 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:02.725369 8576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:02.725522 8576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:02.726634 8576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "279205e0308545cb9cd0aa7436aeb266"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "0b33835f812774b1e4177cb430c147eb"
server_key_iv: "f05710242a8b3eefbe22cf569235235f"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.726985 8576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "279205e0308545cb9cd0aa7436aeb266"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "0b33835f812774b1e4177cb430c147eb"
server_key_iv: "f05710242a8b3eefbe22cf569235235f"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.728298 8576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:36:02.728991 8596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.729169 8576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:36:02.729272 8576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "279205e0308545cb9cd0aa7436aeb266"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "0b33835f812774b1e4177cb430c147eb"
server_key_iv: "f05710242a8b3eefbe22cf569235235f"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.729360 8576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:02.750757 8576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:02.753141 8576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:02.753322 8576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:02.753561 8576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:02.753901 8576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:02.753965 8576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.754009 8576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:02.754036 8576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.760128 8576 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:33163
I20260807 22:36:02.760217 8761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:33163 every 8 connection(s)
I20260807 22:36:02.760560 8576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:36:02.764595 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 8576
I20260807 22:36:02.764739 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:36:02.764961 29003 external_mini_cluster.cc:1428] Setting key 2119a975ab0d5e9bce3d569e1aeb6dc1
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:36:02.770431 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:02.761251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:39965 (local address 127.28.82.254:46879)
0807 22:36:02.761319 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:02.761321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:02.761597 (+ 276us) server_negotiation.cc:407] Connection header received
0807 22:36:02.762118 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:02.762120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:02.762133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:02.762203 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:02.763128 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.763632 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.764767 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.765064 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.765116 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:02.767591 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:02.767593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:02.767594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:02.767602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:02.769025 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:02.769453 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:02.769455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:02.769455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:02.769521 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:02.769789 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:02.769790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:02.769791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:02.769906 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:02.769983 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:02.770258 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 22:36:02.770339 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:36:02.770638 8764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:02.761242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:39965)
0807 22:36:02.761443 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:02.761501 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:02.762006 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:02.762273 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:02.762282 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:02.762846 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:02.763026 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:02.763031 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.763736 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.763737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:02.764651 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:02.764653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.765213 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.765214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:02.765380 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:02.765880 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:02.765891 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:02.767492 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:02.769132 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:02.769139 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:02.769141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:02.769372 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:02.769589 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:02.769590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:02.769590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:02.769706 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:02.769973 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:02.769980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:02.770115 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 22:36:02.770221 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 22:36:02.770384 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":110,"threads_started":1}
I20260807 22:36:02.771037 8762 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46879
I20260807 22:36:02.771129 8762 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:02.771303 8762 heartbeater.cc:511] Master 127.28.82.254:46879 requested a full tablet report, sending...
I20260807 22:36:02.771643 8301 ts_manager.cc:194] Registered new tserver with Master: 279205e0308545cb9cd0aa7436aeb266 (127.28.82.194:33163)
I20260807 22:36:02.772024 8301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:39965
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:36:02.793243 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46879
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:02.877704 8769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:02.877910 8769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:02.877934 8769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:02.879225 8769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:02.879269 8769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:02.879295 8769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:36:02.881166 8769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:02.881357 8769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:02.881526 8769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:02.886624 8777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:02.886777 8769 server_base.cc:1034] running on GCE node
W20260807 22:36:02.886626 8778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:02.886619 8782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:02.887187 8769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:02.887486 8769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:02.888659 8769 hybrid_clock.cc:648] HybridClock initialized: now 1786142162888630 us; error 41 us; skew 500 ppm
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:02.891705 8769 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:36:02.892124 8769 webserver.cc:460] Webserver started at http://127.28.82.195:40417/ using document root <none> and password file <none>
I20260807 22:36:02.892357 8769 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:02.892401 8769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:02.892547 8769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:02.893658 8769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "1f19e3e423dd42449f73861b4bb06d67"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "d7b761cb58578711fe869e8fb5b35e56"
server_key_iv: "95f6817444c863dd15f990c108d93351"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.894068 8769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "1f19e3e423dd42449f73861b4bb06d67"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "d7b761cb58578711fe869e8fb5b35e56"
server_key_iv: "95f6817444c863dd15f990c108d93351"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.895464 8769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:36:02.896176 8792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.896358 8769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:36:02.896448 8769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "1f19e3e423dd42449f73861b4bb06d67"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "d7b761cb58578711fe869e8fb5b35e56"
server_key_iv: "95f6817444c863dd15f990c108d93351"
server_key_version: "encryptionkey@0"
I20260807 22:36:02.896502 8769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:02.913125 8769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:02.915951 8769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:02.916157 8769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:02.916412 8769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:02.916755 8769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:02.916795 8769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.916853 8769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:02.916908 8769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:02.923327 8769 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:42713
I20260807 22:36:02.923420 8964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:42713 every 8 connection(s)
I20260807 22:36:02.923753 8769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:36:02.928999 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 8769
I20260807 22:36:02.929128 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:36:02.929320 29003 external_mini_cluster.cc:1428] Setting key fd9d4be1727dad3bd4acb4a59f99747c
Aug 07 22:36:02 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142162, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:36:02.933893 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:02.924494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:33247 (local address 127.28.82.254:46879)
0807 22:36:02.924553 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:02.924555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:02.924767 (+ 212us) server_negotiation.cc:407] Connection header received
0807 22:36:02.925394 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:02.925395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:02.925419 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:02.925476 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:02.926551 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.927055 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.928164 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.928357 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.928420 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:02.931038 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:02.931039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:02.931039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:02.931046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:02.932556 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:02.932995 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:02.932996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:02.932996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:02.933049 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:02.933364 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:02.933365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:02.933365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:02.933468 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:02.933522 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:02.933766 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 22:36:02.933828 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 22:36:02.934059 8967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:02.924488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:33247)
0807 22:36:02.924638 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:02.924674 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:02.925181 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:02.925596 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:02.925600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:02.926227 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:02.926408 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:02.926413 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.927168 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.927169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:02.928063 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:02.928066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:02.928444 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:02.928445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:02.928550 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:02.929045 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:02.929057 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:02.930948 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:02.932651 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:02.932657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:02.932659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:02.932881 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:02.933141 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:02.933141 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:02.933142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:02.933289 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:02.933528 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:02.933533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:02.933653 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 22:36:02.933738 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 22:36:02.933855 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260807 22:36:02.934558 8965 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46879
I20260807 22:36:02.934707 8965 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:02.934895 8965 heartbeater.cc:511] Master 127.28.82.254:46879 requested a full tablet report, sending...
I20260807 22:36:02.935284 8301 ts_manager.cc:194] Registered new tserver with Master: 1f19e3e423dd42449f73861b4bb06d67 (127.28.82.195:42713)
I20260807 22:36:02.935704 8301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:33247
I20260807 22:36:02.941242 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab.
Entry for principal oryx/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab.
WARNING: no policy specified for oryx/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab.
Entry for principal oryx/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab.
WARNING: no policy specified for oryx/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab.
Entry for principal oryx/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab.
I20260807 22:36:03.017617 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 8394
I20260807 22:36:03.033223 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 8576
I20260807 22:36:03.042984 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 8769
I20260807 22:36:03.051702 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 8256
I20260807 22:36:03.063086 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46879
--webserver_interface=127.28.82.254
--webserver_port=34231
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:46879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:03.145802 8978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:03.146003 8978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:03.146024 8978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:03.147120 8978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:36:03.147163 8978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:03.147178 8978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:03.147190 8978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:36:03.147202 8978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:36:03.148591 8978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:46879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=34231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:03.148769 8978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:03.148940 8978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:03.154102 8988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:03.154130 8984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:03.154171 8983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:03.154563 8978 server_base.cc:1034] running on GCE node
I20260807 22:36:03.154856 8978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:03.155181 8978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:03.156349 8978 hybrid_clock.cc:648] HybridClock initialized: now 1786142163156332 us; error 53 us; skew 500 ppm
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142163, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:03.159078 8978 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:36:03.159456 8978 webserver.cc:460] Webserver started at http://127.28.82.254:34231/ using document root <none> and password file <none>
I20260807 22:36:03.159713 8978 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:03.159775 8978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:03.161166 8978 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:03.161916 8998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.162101 8978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:03.162207 8978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "f4e045978f0246f99aa851aa21d557f5"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "ba1c5d31d0710367254996b39a7fb8d4"
server_key_iv: "1f3ffc258e527938ea656fa9a5e1b65e"
server_key_version: "encryptionkey@0"
I20260807 22:36:03.162590 8978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:03.178400 8978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:03.180310 8978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:03.180480 8978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:03.184621 8978 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:46879
I20260807 22:36:03.184696 9078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:46879 every 8 connection(s)
I20260807 22:36:03.185070 8978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:36:03.186720 9079 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:36:03.187453 9079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Bootstrap starting.
I20260807 22:36:03.188378 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 8978
I20260807 22:36:03.188696 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:40769
--local_ip_for_outbound_sockets=127.28.82.193
--tserver_master_addrs=127.28.82.254:46879
--webserver_port=40989
--webserver_interface=127.28.82.193
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.82.193
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:36:03.197366 9079 log.cc:824] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:36:03.199265 9079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:36:03.199604 9079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Bootstrap complete.
I20260807 22:36:03.202091 9079 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } }
I20260807 22:36:03.202333 9079 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e045978f0246f99aa851aa21d557f5, State: Initialized, Role: FOLLOWER
I20260807 22:36:03.202534 9079 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } }
I20260807 22:36:03.202625 9079 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:36:03.202682 9079 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:36:03.202736 9079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:36:03.203809 9079 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } }
I20260807 22:36:03.203931 9079 leader_election.cc:302] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4e045978f0246f99aa851aa21d557f5; no voters:
I20260807 22:36:03.204183 9079 leader_election.cc:288] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:36:03.204234 9085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:36:03.204382 9085 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [term 2 LEADER]: Becoming Leader. State: Replica: f4e045978f0246f99aa851aa21d557f5, State: Running, Role: LEADER
I20260807 22:36:03.204483 9079 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:36:03.204507 9085 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } }
I20260807 22:36:03.204807 9085 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4e045978f0246f99aa851aa21d557f5. Latest consensus state: current_term: 2 leader_uuid: "f4e045978f0246f99aa851aa21d557f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } } }
I20260807 22:36:03.204903 9085 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:03.204813 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f4e045978f0246f99aa851aa21d557f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4e045978f0246f99aa851aa21d557f5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46879 } } }
I20260807 22:36:03.205093 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:03.205703 9095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:36:03.206311 9095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:36:03.206441 9095 catalog_manager.cc:1259] Loaded cluster ID: 2c480cf8963b4ede827bec7bf7880183
I20260807 22:36:03.206497 9095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:36:03.207710 9095 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:36:03.207906 9095 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f4e045978f0246f99aa851aa21d557f5: Loaded TSK: 0
I20260807 22:36:03.208125 9095 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:36:03.283697 9082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:03.283934 9082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:03.283975 9082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:03.285211 9082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:03.285277 9082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:03.285321 9082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:36:03.286837 9082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.193.keytab
--principal=oryx/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:40769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=40989
--tserver_master_addrs=127.28.82.254:46879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:03.287103 9082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:03.287358 9082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:03.292364 9126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:03.292397 9122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:03.292384 9121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:03.292965 9082 server_base.cc:1034] running on GCE node
I20260807 22:36:03.293175 9082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:03.293431 9082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:03.294551 9082 hybrid_clock.cc:648] HybridClock initialized: now 1786142163294530 us; error 34 us; skew 500 ppm
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142163, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:03.297312 9082 init.cc:377] Logged in from keytab as oryx/127.28.82.193@KRBTEST.COM (short username oryx)
I20260807 22:36:03.297842 9082 webserver.cc:460] Webserver started at http://127.28.82.193:40989/ using document root <none> and password file <none>
I20260807 22:36:03.298111 9082 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:03.298194 9082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:03.299994 9082 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:03.300724 9140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.300972 9082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:03.301074 9082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "ad8733f3688c4c058553e0e73d148c22"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "0e1c298eac80ed4034b00a630bd8d652"
server_key_iv: "4583f2d9445c203f012e0d8b1c8ec65f"
server_key_version: "encryptionkey@0"
I20260807 22:36:03.301489 9082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:03.337958 9082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:03.340547 9082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:03.340706 9082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:03.340961 9082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:03.341300 9082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:03.341343 9082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.341377 9082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:03.341394 9082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.348658 9082 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:40769
I20260807 22:36:03.348745 9304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:40769 every 8 connection(s)
I20260807 22:36:03.349109 9082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:36:03.355667 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 9082
I20260807 22:36:03.356004 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:33163
--local_ip_for_outbound_sockets=127.28.82.194
--tserver_master_addrs=127.28.82.254:46879
--webserver_port=43311
--webserver_interface=127.28.82.194
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.82.194
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:03.358554 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.349966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:37261)
0807 22:36:03.350153 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.350196 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.350793 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.351237 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.351241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.351905 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.352101 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.352106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.353214 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.353215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.353983 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.353985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.354681 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.354682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.354817 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.355384 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.355393 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.358300 (+ 2907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
W20260807 22:36:03.358800 9305 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:46879 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.82.254:46879: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
W20260807 22:36:03.358912 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.350065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:37261 (local address 127.28.82.254:46879)
0807 22:36:03.350286 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.350294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.350336 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:36:03.351004 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.351010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.351062 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.351198 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.352232 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.353105 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.354102 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.354576 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.354705 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.358421 (+ 3716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:37261: BlockingRecv error: recv got EOF from 127.28.82.193:37261 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":119,"threads_started":1}
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:03.364017 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.359052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:39949)
0807 22:36:03.359171 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.359182 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.359258 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.359406 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.359407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.359571 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.359623 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.359626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.360292 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.360294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.361171 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.361174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.361546 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.361547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.361646 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.362043 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.362049 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.363935 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102}
W20260807 22:36:03.364071 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.359120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:39949 (local address 127.28.82.254:46879)
0807 22:36:03.359155 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.359158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.359243 (+ 85us) server_negotiation.cc:407] Connection header received
0807 22:36:03.359318 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.359319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.359330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.359403 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.359738 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.360195 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.361264 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.361445 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.361533 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.364010 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:39949: BlockingRecv error: recv got EOF from 127.28.82.193:39949 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:03.369149 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.364438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:33849)
0807 22:36:03.364556 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.364568 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.364624 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.364769 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.364770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.364937 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.364973 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.364975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.365642 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.365644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.366543 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.366547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.366887 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.366888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.366951 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.367338 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.367343 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.369072 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260807 22:36:03.369191 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.364504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:33849 (local address 127.28.82.254:46879)
0807 22:36:03.364558 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.364559 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.364613 (+ 54us) server_negotiation.cc:407] Connection header received
0807 22:36:03.364687 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.364687 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.364696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.364741 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.365051 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.365556 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.366624 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.366806 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.366862 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.369138 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:33849: BlockingRecv error: recv got EOF from 127.28.82.193:33849 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 22:36:03.369364 9305 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:36:03.442720 9311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:03.442929 9311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:03.442955 9311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:03.444141 9311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:03.444190 9311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:03.444218 9311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:36:03.445664 9311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.194.keytab
--principal=oryx/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:33163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=43311
--tserver_master_addrs=127.28.82.254:46879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:03.445905 9311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:03.446172 9311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:03.451418 9318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:03.451424 9321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:03.451426 9317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:03.451710 9311 server_base.cc:1034] running on GCE node
I20260807 22:36:03.451980 9311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:03.452256 9311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:03.453423 9311 hybrid_clock.cc:648] HybridClock initialized: now 1786142163453414 us; error 31 us; skew 500 ppm
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142163, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:03.456043 9311 init.cc:377] Logged in from keytab as oryx/127.28.82.194@KRBTEST.COM (short username oryx)
I20260807 22:36:03.456472 9311 webserver.cc:460] Webserver started at http://127.28.82.194:43311/ using document root <none> and password file <none>
I20260807 22:36:03.456722 9311 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:03.456782 9311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:03.458178 9311 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:03.458894 9329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.459098 9311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:36:03.459179 9311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "279205e0308545cb9cd0aa7436aeb266"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "0b33835f812774b1e4177cb430c147eb"
server_key_iv: "f05710242a8b3eefbe22cf569235235f"
server_key_version: "encryptionkey@0"
I20260807 22:36:03.459568 9311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:03.469646 9311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:03.471967 9311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:03.472080 9311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:03.472266 9311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:03.472525 9311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:03.472555 9311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.472579 9311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:03.472591 9311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.478569 9311 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:33163
I20260807 22:36:03.478698 9487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:33163 every 8 connection(s)
I20260807 22:36:03.479007 9311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:36:03.482962 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 9311
I20260807 22:36:03.483276 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:42713
--local_ip_for_outbound_sockets=127.28.82.195
--tserver_master_addrs=127.28.82.254:46879
--webserver_port=40417
--webserver_interface=127.28.82.195
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.82.195
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:36:03.487547 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.479821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:56449 (local address 127.28.82.254:46879)
0807 22:36:03.479867 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.479869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.480116 (+ 247us) server_negotiation.cc:407] Connection header received
0807 22:36:03.480584 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.480585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.480602 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.480676 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.481656 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.482153 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.483576 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.483775 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.483835 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.487489 (+ 3654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:56449: BlockingRecv error: recv got EOF from 127.28.82.194:56449 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 22:36:03.487607 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.479769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:56449)
0807 22:36:03.479941 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.479984 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.480466 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.480768 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.480772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.481348 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.481535 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.481540 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.482236 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.482236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.483441 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.483444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.483865 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.483867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.484018 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.484552 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.484564 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.487406 (+ 2842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
W20260807 22:36:03.487831 9490 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:46879 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.82.254:46879: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:03.492676 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.488094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:33639)
0807 22:36:03.488151 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.488160 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.488215 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.488342 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.488342 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.488478 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.488523 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.488524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.489291 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.489292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.489793 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.489795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.490162 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.490163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.490232 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.490727 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.490735 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.492606 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:36:03.492780 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.488160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:33639 (local address 127.28.82.254:46879)
0807 22:36:03.488223 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.488225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.488250 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:36:03.488274 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.488275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.488284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.488335 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.488600 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.489207 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.489879 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.490086 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.490164 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.492714 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:33639: BlockingRecv error: recv got EOF from 127.28.82.194:33639 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:03.497844 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.493020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:34283)
0807 22:36:03.493159 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.493191 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.493253 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.493393 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.493394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.493553 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.493589 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.493591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.494238 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.494239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.495144 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.495147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.495499 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.495500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.495561 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.495984 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.495989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.497775 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119}
W20260807 22:36:03.497951 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.493103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:34283 (local address 127.28.82.254:46879)
0807 22:36:03.493160 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.493161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.493271 (+ 110us) server_negotiation.cc:407] Connection header received
0807 22:36:03.493302 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.493302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.493312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.493366 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.493719 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.494154 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.495231 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.495419 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.495477 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.497864 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:34283: BlockingRecv error: recv got EOF from 127.28.82.194:34283 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 22:36:03.497959 9490 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:36:03.570742 9496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:03.570940 9496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:03.570963 9496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:03.572042 9496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:03.572078 9496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:03.572103 9496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:36:03.573433 9496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/krb5kdc/oryx_127.28.82.195.keytab
--principal=oryx/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:42713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=40417
--tserver_master_addrs=127.28.82.254:46879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:03.573606 9496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:03.573833 9496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:03.578851 9503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:03.578888 9501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:03.579011 9496 server_base.cc:1034] running on GCE node
W20260807 22:36:03.578852 9505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:03.579311 9496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:03.579566 9496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:03.580750 9496 hybrid_clock.cc:648] HybridClock initialized: now 1786142163580726 us; error 39 us; skew 500 ppm
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142163, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:03.583386 9496 init.cc:377] Logged in from keytab as oryx/127.28.82.195@KRBTEST.COM (short username oryx)
I20260807 22:36:03.583753 9496 webserver.cc:460] Webserver started at http://127.28.82.195:40417/ using document root <none> and password file <none>
I20260807 22:36:03.583930 9496 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:03.583981 9496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:03.585299 9496 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:03.586012 9515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.586241 9496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:03.586349 9496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "1f19e3e423dd42449f73861b4bb06d67"
format_stamp: "Formatted at 2026-08-07 22:36:02 on dist-test-slave-7kzw"
server_key: "d7b761cb58578711fe869e8fb5b35e56"
server_key_iv: "95f6817444c863dd15f990c108d93351"
server_key_version: "encryptionkey@0"
I20260807 22:36:03.586767 9496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:03.606216 9496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:03.608819 9496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:03.609002 9496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:03.609283 9496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:03.609634 9496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:03.609696 9496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.609743 9496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:03.609776 9496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:03.616250 9496 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:42713
I20260807 22:36:03.616302 9684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:42713 every 8 connection(s)
I20260807 22:36:03.616677 9496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:36:03.620518 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 9496
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 22:36:03.625653 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.617618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:53611 (local address 127.28.82.254:46879)
0807 22:36:03.617706 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.617709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.617959 (+ 250us) server_negotiation.cc:407] Connection header received
0807 22:36:03.618427 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.618429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.618441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.618681 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.619679 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.620178 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.621609 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.621827 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.621889 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.625551 (+ 3662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:53611: BlockingRecv error: recv got EOF from 127.28.82.195:53611 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:36:03.625660 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.617578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:53611)
0807 22:36:03.617824 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.617862 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.618318 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.618770 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.618773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.619366 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.619563 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.619568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.620267 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.620268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.621517 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.621519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.621919 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.621920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.622085 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.622739 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.622748 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.625438 (+ 2690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":123,"threads_started":1}
W20260807 22:36:03.625933 9687 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:46879 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.82.254:46879: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
I20260807 22:36:03.629861 9694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.621128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.28.82.254:46879)
0807 22:36:03.621260 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.621261 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.621342 (+ 81us) server_negotiation.cc:407] Connection header received
0807 22:36:03.621421 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.621422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.621432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.621494 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.622250 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.622956 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.623910 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.624139 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.624212 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.625658 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:03.625662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:03.625662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:03.625676 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:03.627986 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:03.628401 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:03.628404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:03.628405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:03.628582 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:03.628905 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:03.628905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:03.628906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:03.629243 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:03.629385 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:03.629600 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 22:36:03.629748 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:03.631431 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.626212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:33681)
0807 22:36:03.626309 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.626320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.626396 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.626607 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.626608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.626740 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.626785 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.626786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.627680 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.627681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.628281 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.628284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.628759 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.628760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.628834 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.629307 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.629312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.631372 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260807 22:36:03.631646 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.626261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:33681 (local address 127.28.82.254:46879)
0807 22:36:03.626298 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.626300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.626381 (+ 81us) server_negotiation.cc:407] Connection header received
0807 22:36:03.626487 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.626488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.626500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.626562 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.626875 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.627572 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.628392 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.628667 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.628741 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.631567 (+ 2826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:33681: BlockingRecv error: recv got EOF from 127.28.82.195:33681 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:03 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:03.635459 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:03.631695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:51783)
0807 22:36:03.631771 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:03.631780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:03.631816 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:03.631968 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:03.631969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:03.632073 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:03.632097 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.632098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.632650 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.632650 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.633145 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:03.633146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.633450 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.633450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:03.633485 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.633836 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:03.633840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:03.635419 (+ 1579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 22:36:03.635581 9687 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:36:03.635591 9308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:03.631742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:51783 (local address 127.28.82.254:46879)
0807 22:36:03.631859 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:03.631860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:03.631869 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:36:03.631895 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:03.631896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:03.631903 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:03.631951 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:03.632160 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.632583 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.633217 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:03.633385 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:03.633436 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:03.635521 (+ 2085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:51783: BlockingRecv error: recv got EOF from 127.28.82.195:51783 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 07 22:36:04 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:04 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:04.374934 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:04.369865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:41295)
0807 22:36:04.370010 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:04.370030 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:04.370112 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:04.370333 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:04.370335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:04.370637 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:04.370708 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:04.370712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.371433 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.371434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:04.372035 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:04.372037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.372378 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.372379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:04.372427 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:04.372897 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:04.372902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:04.374865 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
W20260807 22:36:04.375034 9698 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:04.369939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:41295 (local address 127.28.82.254:46879)
0807 22:36:04.370101 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:04.370103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:04.370114 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:36:04.370210 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:04.370211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:04.370224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:04.370303 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:04.370808 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.371356 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.372111 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.372304 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.372356 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:04.374959 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:41295: BlockingRecv error: recv got EOF from 127.28.82.193:41295 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":112,"threads_started":1}
Aug 07 22:36:04 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:04 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:04.503643 9699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:04.498337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:56697)
0807 22:36:04.498540 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:04.498554 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:04.498681 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:04.498842 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:04.498843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:04.499030 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:04.499077 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:04.499079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.499818 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.499819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:04.500412 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:04.500414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.500807 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.500808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:04.500876 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:04.501357 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:04.501362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:04.503566 (+ 2204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":86,"threads_started":1}
W20260807 22:36:04.503805 9698 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:04.498449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:56697 (local address 127.28.82.254:46879)
0807 22:36:04.498544 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:04.498546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:04.498658 (+ 112us) server_negotiation.cc:407] Connection header received
0807 22:36:04.498743 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:04.498744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:04.498758 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:04.498834 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:04.499170 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.499725 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.500509 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.500718 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.500806 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:04.503719 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:56697: BlockingRecv error: recv got EOF from 127.28.82.194:56697 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 22:36:04 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:04 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:04.640790 9700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:04.636012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:54353)
0807 22:36:04.636174 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:04.636186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:04.636258 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:04.636390 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:04.636391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:04.636560 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:04.636607 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:04.636609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.637256 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.637256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:04.637785 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:04.637787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.638110 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.638111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:04.638148 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:04.638765 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:04.638771 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:04.640724 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
W20260807 22:36:04.640882 9698 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:04.636078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:54353 (local address 127.28.82.254:46879)
0807 22:36:04.636187 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:04.636189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:04.636245 (+ 56us) server_negotiation.cc:407] Connection header received
0807 22:36:04.636306 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:04.636306 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:04.636319 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:04.636370 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:04.636674 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.637165 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.637862 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:04.638046 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:04.638099 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:04.640814 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:54353: BlockingRecv error: recv got EOF from 127.28.82.195:54353 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 22:36:05 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:05 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:05.380422 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:05.375463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:39293)
0807 22:36:05.375607 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:05.375618 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:05.375682 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:05.375799 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:05.375800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:05.375973 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:05.376023 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:05.376025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.376731 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.376731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:05.377255 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:05.377256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.377647 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.377648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:05.377688 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:05.378301 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:05.378308 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:05.380359 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":136,"threads_started":1}
W20260807 22:36:05.380534 9705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:05.375571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:39293 (local address 127.28.82.254:46879)
0807 22:36:05.375680 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:05.375682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:05.375693 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:36:05.375725 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:05.375725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:05.375736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:05.375795 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:05.376103 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.376653 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.377332 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.377529 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.377629 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:05.380454 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:39293: BlockingRecv error: recv got EOF from 127.28.82.193:39293 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
Aug 07 22:36:05 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:05 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:05.509141 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:05.504138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:45775)
0807 22:36:05.504288 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:05.504300 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:05.504385 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:05.504517 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:05.504518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:05.504779 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:05.504829 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:05.504831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.505600 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.505601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:05.506182 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:05.506184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.506568 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.506569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:05.506609 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:05.507163 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:05.507169 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:05.509081 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":57,"threads_started":1}
W20260807 22:36:05.509264 9705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:05.504251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:45775 (local address 127.28.82.254:46879)
0807 22:36:05.504300 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:05.504302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:05.504392 (+ 90us) server_negotiation.cc:407] Connection header received
0807 22:36:05.504436 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:05.504436 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:05.504448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:05.504497 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:05.504905 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.505529 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.506256 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.506441 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.506575 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:05.509188 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:45775: BlockingRecv error: recv got EOF from 127.28.82.194:45775 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 22:36:05 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:05 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:05.646400 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:05.641321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:57033)
0807 22:36:05.641468 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:05.641504 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:05.641575 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:05.641792 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:05.641794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:05.641984 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:05.642037 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:05.642039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.642772 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.642773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:05.643303 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:05.643305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.643656 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.643657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:05.643719 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:05.644277 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:05.644282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:05.646303 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
W20260807 22:36:05.646446 9705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:05.641462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:57033 (local address 127.28.82.254:46879)
0807 22:36:05.641530 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:05.641532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:05.641555 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:36:05.641655 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:05.641656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:05.641676 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:05.641769 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:05.642121 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.642681 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.643392 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:05.643581 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:05.643657 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:05.646379 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:57033: BlockingRecv error: recv got EOF from 127.28.82.195:57033 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 22:36:06 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:06 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:06.389485 9709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:06.380951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:47165)
0807 22:36:06.381110 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:06.381122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:06.381187 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:06.381339 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:06.381340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:06.381511 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:06.381563 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:06.381564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.382266 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.382267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:06.382813 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:06.382815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.383194 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.383194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:06.383234 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:06.387043 (+ 3809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:06.387049 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:06.389380 (+ 2331us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
W20260807 22:36:06.389644 9710 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:06.381067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:47165 (local address 127.28.82.254:46879)
0807 22:36:06.381207 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:06.381209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:06.381218 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:36:06.381242 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:06.381242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:06.381253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:06.381317 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:06.381644 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.382190 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.382909 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.383112 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.383180 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:06.389558 (+ 6378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:47165: BlockingRecv error: recv got EOF from 127.28.82.193:47165 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
Aug 07 22:36:06 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:06 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:06.514286 9712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:06.509669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:55509)
0807 22:36:06.509834 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:06.509856 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:06.509923 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:06.510090 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:06.510091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:06.510273 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:06.510321 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:06.510322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.510985 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.510986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:06.511517 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:06.511519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.511848 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.511849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:06.511894 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:06.512411 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:06.512416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:06.514216 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
W20260807 22:36:06.514382 9710 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:06.509744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:55509 (local address 127.28.82.254:46879)
0807 22:36:06.509837 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:06.509843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:06.509922 (+ 79us) server_negotiation.cc:407] Connection header received
0807 22:36:06.509970 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:06.509970 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:06.509981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:06.510035 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:06.510390 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.510912 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.511586 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.511782 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.511833 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:06.514310 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:55509: BlockingRecv error: recv got EOF from 127.28.82.194:55509 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 22:36:06 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:06 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:06.651638 9714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:06.646849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:46339)
0807 22:36:06.646997 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:06.647012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:06.647087 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:06.647216 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:06.647217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:06.647402 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:06.647451 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:06.647453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.648187 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.648187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:06.648718 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:06.648719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.649086 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.649087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:06.649135 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:06.649661 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:06.649666 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:06.651568 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
W20260807 22:36:06.651808 9710 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:06.646924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:46339 (local address 127.28.82.254:46879)
0807 22:36:06.647010 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:06.647012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:06.647073 (+ 61us) server_negotiation.cc:407] Connection header received
0807 22:36:06.647138 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:06.647139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:06.647152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:06.647193 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:06.647591 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.648098 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.648809 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:06.649002 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:06.649071 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:06.651716 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:46339: BlockingRecv error: recv got EOF from 127.28.82.195:46339 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 22:36:07 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:07 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:07.395072 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:07.390034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:47293)
0807 22:36:07.390164 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:07.390177 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:07.390288 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:07.390445 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:07.390446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:07.390633 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:07.390685 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:07.390687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.391469 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.391470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:07.392170 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:07.392173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.392549 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.392550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:07.392592 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:07.393051 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:07.393056 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:07.394973 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
W20260807 22:36:07.395131 9719 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:07.390124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:47293 (local address 127.28.82.254:46879)
0807 22:36:07.390294 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:07.390296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:07.390305 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:36:07.390350 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:07.390350 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:07.390360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:07.390422 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:07.390785 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.391341 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.392265 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.392475 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.392535 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:07.395055 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:47293: BlockingRecv error: recv got EOF from 127.28.82.193:47293 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
Aug 07 22:36:07 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:07 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:07.519577 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:07.514838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:59269)
0807 22:36:07.514994 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:07.515006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:07.515077 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:07.515232 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:07.515232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:07.515405 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:07.515455 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:07.515457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.516132 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.516133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:07.516666 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:07.516668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.517049 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.517050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:07.517100 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:07.517594 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:07.517599 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:07.519516 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
W20260807 22:36:07.519810 9719 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:07.514914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:59269 (local address 127.28.82.254:46879)
0807 22:36:07.514990 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:07.514993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:07.515072 (+ 79us) server_negotiation.cc:407] Connection header received
0807 22:36:07.515149 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:07.515150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:07.515162 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:07.515215 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:07.515517 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.516044 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.516768 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.516973 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.517039 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:07.519731 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:59269: BlockingRecv error: recv got EOF from 127.28.82.194:59269 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 22:36:07 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:07 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:07.657233 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:07.652309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:43951)
0807 22:36:07.652470 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:07.652482 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:07.652548 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:07.652667 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:07.652668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:07.652890 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:07.652959 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:07.652962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.653851 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.653852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:07.654406 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:07.654407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.654805 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.654806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:07.654859 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:07.655307 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:07.655312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:07.657172 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
W20260807 22:36:07.657351 9719 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:07.652409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:43951 (local address 127.28.82.254:46879)
0807 22:36:07.652460 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:07.652462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:07.652552 (+ 90us) server_negotiation.cc:407] Connection header received
0807 22:36:07.652597 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:07.652597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:07.652611 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:07.652662 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:07.653059 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.653774 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.654517 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:07.654715 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:07.654769 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:07.657270 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:43951: BlockingRecv error: recv got EOF from 127.28.82.195:43951 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 07 22:36:08 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:08 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:08.404379 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:08.395584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:56945)
0807 22:36:08.395777 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:08.395789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:08.395844 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:08.395963 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:08.395964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:08.396142 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:08.396195 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:08.396197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.396885 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.396887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:08.397439 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:08.397441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.397913 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.397915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:08.397970 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:08.402383 (+ 4413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:08.402394 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:08.404313 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
W20260807 22:36:08.404551 9723 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:08.395714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:56945 (local address 127.28.82.254:46879)
0807 22:36:08.395852 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:08.395853 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:08.395863 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:36:08.395886 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:08.395887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:08.395898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:08.395952 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:08.396265 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.396790 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.397541 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.397810 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.397878 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:08.404445 (+ 6567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:56945: BlockingRecv error: recv got EOF from 127.28.82.193:56945 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
Aug 07 22:36:08 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:08 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:08.525110 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:08.520081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:56459)
0807 22:36:08.520231 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:08.520244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:08.520324 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:08.520474 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:08.520475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:08.520650 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:08.520700 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:08.520701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.521352 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.521353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:08.521880 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:08.521881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.522261 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.522262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:08.522307 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:08.522813 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:08.522818 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:08.525040 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":59,"threads_started":1}
W20260807 22:36:08.525305 9723 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:08.520262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:56459 (local address 127.28.82.254:46879)
0807 22:36:08.520323 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:08.520326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:08.520360 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:36:08.520389 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:08.520389 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:08.520403 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:08.520454 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:08.520767 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.521278 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.521981 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.522179 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.522249 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:08.525201 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:56459: BlockingRecv error: recv got EOF from 127.28.82.194:56459 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 22:36:08 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:08 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:08.663563 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:08.657792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:52775)
0807 22:36:08.657982 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:08.657995 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:08.658060 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:08.658243 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:08.658245 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:08.658432 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:08.658547 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:08.658549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.659279 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.659280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:08.659877 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:08.659879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.660330 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.660331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:08.660388 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:08.661026 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:08.661032 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:08.663462 (+ 2430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":88,"threads_started":1}
W20260807 22:36:08.663619 9723 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:08.657939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:52775 (local address 127.28.82.254:46879)
0807 22:36:08.658041 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:08.658043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:08.658056 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:36:08.658140 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:08.658141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:08.658157 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:08.658206 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:08.658650 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.659176 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.660025 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:08.660242 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:08.660314 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:08.663545 (+ 3231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:52775: BlockingRecv error: recv got EOF from 127.28.82.195:52775 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 22:36:09 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:09 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:09.410283 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:09.404897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:34423)
0807 22:36:09.405062 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:09.405076 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:09.405185 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:09.405390 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:09.405391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:09.405655 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:09.405746 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:09.405748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.406589 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.406591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:09.407204 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:09.407206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.407653 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.407654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:09.407756 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:09.408339 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:09.408343 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:09.410222 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
W20260807 22:36:09.410446 9729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:09.404979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:34423 (local address 127.28.82.254:46879)
0807 22:36:09.405170 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:09.405172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:09.405184 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:36:09.405257 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:09.405257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:09.405270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:09.405384 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:09.405808 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.406447 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.407288 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.407528 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.407600 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:09.410349 (+ 2749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:34423: BlockingRecv error: recv got EOF from 127.28.82.193:34423 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
Aug 07 22:36:09 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:09 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:09.531140 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:09.525604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:32915)
0807 22:36:09.525803 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:09.525820 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:09.525925 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:09.526114 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:09.526115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:09.526383 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:09.526449 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:09.526453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.527336 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.527337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:09.527908 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:09.527910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.528314 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.528315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:09.528365 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:09.528961 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:09.528967 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:09.531041 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
W20260807 22:36:09.531213 9729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:09.525748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:32915 (local address 127.28.82.254:46879)
0807 22:36:09.525822 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:09.525824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:09.525903 (+ 79us) server_negotiation.cc:407] Connection header received
0807 22:36:09.526006 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:09.526008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:09.526034 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:09.526105 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:09.526603 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.527247 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.528020 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.528227 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.528292 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:09.531135 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:32915: BlockingRecv error: recv got EOF from 127.28.82.194:32915 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 22:36:09 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:09 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:09.670625 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:09.664118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:37207)
0807 22:36:09.664289 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:09.664304 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:09.664394 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:09.664587 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:09.664588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:09.664817 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:09.664868 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:09.664870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.666131 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.666132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:09.666964 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:09.666968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.667598 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.667600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:09.667664 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:09.668233 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:09.668239 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:09.670507 (+ 2268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
W20260807 22:36:09.670665 9729 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:09.664191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:37207 (local address 127.28.82.254:46879)
0807 22:36:09.664293 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:09.664296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:09.664389 (+ 93us) server_negotiation.cc:407] Connection header received
0807 22:36:09.664459 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:09.664459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:09.664475 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:09.664568 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:09.665206 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.666023 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.667123 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:09.667469 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:09.667572 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:09.670588 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:37207: BlockingRecv error: recv got EOF from 127.28.82.195:37207 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 22:36:10 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:10 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:10.415648 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:10.410788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:60361)
0807 22:36:10.410983 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:10.410994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:10.411057 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:10.411210 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:10.411211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:10.411388 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:10.411439 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:10.411441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.412177 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.412178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:10.412696 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:10.412697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.413144 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.413145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:10.413206 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:10.413645 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:10.413649 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:10.415558 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":128,"threads_started":1}
W20260807 22:36:10.415810 9739 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:10.410884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:60361 (local address 127.28.82.254:46879)
0807 22:36:10.411013 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:10.411015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:10.411065 (+ 50us) server_negotiation.cc:407] Connection header received
0807 22:36:10.411113 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:10.411114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:10.411126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:10.411192 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:10.411515 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.412099 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.412792 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.413062 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.413144 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:10.415737 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:60361: BlockingRecv error: recv got EOF from 127.28.82.193:60361 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
Aug 07 22:36:10 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:10 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:10.537066 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:10.531828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:55521)
0807 22:36:10.531980 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:10.531993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:10.532104 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:10.532301 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:10.532303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:10.532508 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:10.532566 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:10.532569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.533320 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.533321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:10.533909 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:10.533911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.534322 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.534323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:10.534391 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:10.534912 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:10.534917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:10.536970 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
W20260807 22:36:10.537146 9739 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:10.531876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:55521 (local address 127.28.82.254:46879)
0807 22:36:10.531950 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:10.531953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:10.532083 (+ 130us) server_negotiation.cc:407] Connection header received
0807 22:36:10.532187 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:10.532188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:10.532206 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:10.532291 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:10.532667 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.533225 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.534010 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.534237 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.534325 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:10.537067 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:55521: BlockingRecv error: recv got EOF from 127.28.82.194:55521 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 22:36:10 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:10 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:10.676115 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:10.671166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:44303)
0807 22:36:10.671367 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:10.671381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:10.671476 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:10.671658 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:10.671660 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:10.671861 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:10.671910 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:10.671912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.672592 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.672593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:10.673248 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:10.673250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.673627 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.673628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:10.673683 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:10.674142 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:10.674147 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:10.676018 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
W20260807 22:36:10.676172 9739 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:10.671287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:44303 (local address 127.28.82.254:46879)
0807 22:36:10.671371 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:10.671374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:10.671455 (+ 81us) server_negotiation.cc:407] Connection header received
0807 22:36:10.671556 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:10.671557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:10.671574 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:10.671648 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:10.671990 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.672496 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.673362 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:10.673551 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:10.673628 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:10.676108 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:44303: BlockingRecv error: recv got EOF from 127.28.82.195:44303 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 22:36:11 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:11 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:11.427126 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:11.416176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:49685)
0807 22:36:11.416358 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:11.416370 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:11.416424 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:11.416709 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:11.416709 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:11.416885 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:11.416936 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:11.416938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.417673 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.417673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:11.418194 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:11.418195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.418700 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.418701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:11.418754 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:11.424937 (+ 6183us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:11.424943 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:11.427030 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
W20260807 22:36:11.427202 9743 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:11.416379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:49685 (local address 127.28.82.254:46879)
0807 22:36:11.416565 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:11.416567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:11.416575 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:36:11.416597 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:11.416597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:11.416612 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:11.416678 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:11.417016 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.417524 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.418319 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.418575 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.418684 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:11.427126 (+ 8442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:49685: BlockingRecv error: recv got EOF from 127.28.82.193:49685 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
Aug 07 22:36:11 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:11 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:11.542677 9744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:11.537641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:43239)
0807 22:36:11.537824 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:11.537837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:11.537903 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:11.538079 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:11.538080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:11.538260 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:11.538311 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:11.538313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.539227 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.539228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:11.539750 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:11.539751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.540142 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.540143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:11.540194 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:11.540746 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:11.540751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:11.542617 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":132,"threads_started":1}
W20260807 22:36:11.542796 9743 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:11.537747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:43239 (local address 127.28.82.254:46879)
0807 22:36:11.537799 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:11.537801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:11.537889 (+ 88us) server_negotiation.cc:407] Connection header received
0807 22:36:11.537959 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:11.537960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:11.537973 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:11.538017 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:11.538402 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.539151 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.539826 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.540021 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.540074 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:11.542725 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:43239: BlockingRecv error: recv got EOF from 127.28.82.194:43239 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 22:36:11 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:11 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:11.681555 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:11.676597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:40215)
0807 22:36:11.676726 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:11.676740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:11.676796 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:11.676968 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:11.676969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:11.677148 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:11.677199 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:11.677200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.677953 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.677954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:11.678513 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:11.678515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.678887 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.678887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:11.678930 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:11.679387 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:11.679391 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:11.681486 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
W20260807 22:36:11.681718 9743 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:11.676733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:40215 (local address 127.28.82.254:46879)
0807 22:36:11.676781 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:11.676783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:11.676794 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:36:11.676859 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:11.676859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:11.676871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:11.676927 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:11.677303 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.677863 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.678601 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:11.678814 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:11.678877 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:11.681583 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:40215: BlockingRecv error: recv got EOF from 127.28.82.195:40215 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 22:36:12 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:12 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:12.432749 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:12.427725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:53447)
0807 22:36:12.427925 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:12.427937 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:12.427999 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:12.428194 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:12.428195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:12.428457 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:12.428509 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:12.428512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.429315 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.429316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:12.429847 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:12.429849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.430225 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.430226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:12.430271 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:12.430753 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:12.430758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:12.432674 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":105,"threads_started":1}
W20260807 22:36:12.432868 9747 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:12.427858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:53447 (local address 127.28.82.254:46879)
0807 22:36:12.427996 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:12.427998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:12.428008 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:36:12.428053 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:12.428054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:12.428065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:12.428128 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:12.428570 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.429242 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.429932 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.430144 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.430201 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:12.432776 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:53447: BlockingRecv error: recv got EOF from 127.28.82.193:53447 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
Aug 07 22:36:12 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:12 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:12.549001 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:12.543265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:44467)
0807 22:36:12.543505 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:12.543520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:12.543608 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:12.543791 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:12.543792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:12.544005 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:12.544066 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:12.544068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.544824 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.544825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:12.545404 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:12.545406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.545889 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.545890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:12.545953 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:12.546581 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:12.546586 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:12.548884 (+ 2298us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":114,"threads_started":1}
W20260807 22:36:12.549041 9747 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:12.543398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:44467 (local address 127.28.82.254:46879)
0807 22:36:12.543486 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:12.543489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:12.543583 (+ 94us) server_negotiation.cc:407] Connection header received
0807 22:36:12.543691 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:12.543692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:12.543707 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:12.543773 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:12.544163 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.544710 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.545512 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.545780 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.545886 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:12.548975 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:44467: BlockingRecv error: recv got EOF from 127.28.82.194:44467 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 22:36:12 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:12 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:12.687099 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:12.681996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:39403)
0807 22:36:12.682281 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:12.682294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:12.682367 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:12.682537 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:12.682538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:12.682775 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:12.682823 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:12.682825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.683600 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.683601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:12.684128 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:12.684130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.684498 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.684499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:12.684572 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:12.685083 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:12.685088 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:12.687000 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":87,"threads_started":1}
W20260807 22:36:12.687153 9747 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:12.682093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:39403 (local address 127.28.82.254:46879)
0807 22:36:12.682270 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:12.682274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:12.682345 (+ 71us) server_negotiation.cc:407] Connection header received
0807 22:36:12.682423 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:12.682424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:12.682440 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:12.682531 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:12.682912 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.683519 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.684217 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:12.684413 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:12.684488 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:12.687087 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:39403: BlockingRecv error: recv got EOF from 127.28.82.195:39403 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 07 22:36:13 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:13 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:13.438491 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:13.433445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:46149)
0807 22:36:13.433610 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:13.433626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:13.433699 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:13.433940 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:13.433941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:13.434131 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:13.434183 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:13.434185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.434996 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.434996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:13.435540 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:13.435542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.435889 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.435890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:13.435929 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:13.436409 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:13.436418 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:13.438410 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
W20260807 22:36:13.438670 9754 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:13.433635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:46149 (local address 127.28.82.254:46879)
0807 22:36:13.433821 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:13.433823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:13.433829 (+ 6us) server_negotiation.cc:407] Connection header received
0807 22:36:13.433851 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:13.433852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:13.433864 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:13.433918 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:13.434308 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.434917 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.435614 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.435820 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.435878 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:13.438595 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:46149: BlockingRecv error: recv got EOF from 127.28.82.193:46149 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
Aug 07 22:36:13 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:13 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:13.554359 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:13.549534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:43483)
0807 22:36:13.549723 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:13.549743 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:13.549791 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:13.549923 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:13.549924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:13.550100 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:13.550150 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:13.550152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.550857 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.550858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:13.551380 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:13.551382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.551710 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.551711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:13.551755 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:13.552312 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:13.552317 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:13.554298 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":126,"threads_started":1}
W20260807 22:36:13.554503 9754 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:13.549720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:43483 (local address 127.28.82.254:46879)
0807 22:36:13.549794 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:13.549796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:13.549807 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:36:13.549836 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:13.549837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:13.549849 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:13.549906 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:13.550243 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.550777 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.551457 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.551644 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.551697 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:13.554399 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:43483: BlockingRecv error: recv got EOF from 127.28.82.194:43483 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 22:36:13 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:13 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:13.692641 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:13.687712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:45459)
0807 22:36:13.687856 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:13.687871 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:13.687940 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:13.688100 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:13.688101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:13.688302 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:13.688350 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:13.688352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.689140 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.689142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:13.689721 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:13.689724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.690120 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.690121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:13.690191 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:13.690700 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:13.690705 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:13.692538 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
W20260807 22:36:13.692725 9754 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:13.687828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:45459 (local address 127.28.82.254:46879)
0807 22:36:13.687883 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:13.687886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:13.687919 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:36:13.688003 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:13.688004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:13.688018 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:13.688093 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:13.688447 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.689034 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.689815 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:13.690048 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:13.690114 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:13.692627 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:45459: BlockingRecv error: recv got EOF from 127.28.82.195:45459 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 07 22:36:14 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:14 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:14.444216 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:14.438995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:36333)
0807 22:36:14.439186 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:14.439198 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:14.439261 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:14.439441 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:14.439442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:14.439653 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:14.439707 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:14.439709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.440835 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.440836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:14.441423 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:14.441426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.441793 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.441794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:14.441844 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:14.442304 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:14.442309 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:14.444134 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":108,"threads_started":1}
W20260807 22:36:14.444344 9758 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:14.439092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:36333 (local address 127.28.82.254:46879)
0807 22:36:14.439236 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:14.439238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:14.439246 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:36:14.439317 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:14.439318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:14.439329 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:14.439379 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:14.439828 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.440729 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.441502 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.441699 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.441760 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:14.444234 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:36333: BlockingRecv error: recv got EOF from 127.28.82.193:36333 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
Aug 07 22:36:14 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:14 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:14.560105 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:14.554995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:33635)
0807 22:36:14.555179 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:14.555191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:14.555255 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:14.555387 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:14.555387 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:14.555569 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:14.555618 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:14.555619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.556310 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.556310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:14.556845 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:14.556847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.557206 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.557207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:14.557254 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:14.557803 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:14.557809 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:14.560034 (+ 2225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
W20260807 22:36:14.560221 9758 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:14.555065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:33635 (local address 127.28.82.254:46879)
0807 22:36:14.555139 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:14.555142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:14.555253 (+ 111us) server_negotiation.cc:407] Connection header received
0807 22:36:14.555302 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:14.555303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:14.555315 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:14.555368 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:14.555716 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.556239 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.556915 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.557131 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.557182 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:14.560133 (+ 2951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:33635: BlockingRecv error: recv got EOF from 127.28.82.194:33635 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 22:36:14 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:14 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:14.698057 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:14.693155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:50799)
0807 22:36:14.693288 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:14.693300 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:14.693354 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:14.693501 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:14.693502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:14.693690 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:14.693738 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:14.693740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.694391 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.694392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:14.694993 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:14.694996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.695385 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.695386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:14.695429 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:14.695960 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:14.695965 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:14.697955 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
W20260807 22:36:14.698104 9758 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:14.693232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:50799 (local address 127.28.82.254:46879)
0807 22:36:14.693324 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:14.693326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:14.693337 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:36:14.693415 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:14.693416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:14.693429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:14.693481 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:14.693803 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.694312 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.695095 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:14.695301 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:14.695376 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:14.698038 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:50799: BlockingRecv error: recv got EOF from 127.28.82.195:50799 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 22:36:15 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:15 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:15.449637 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:15.444733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:38525)
0807 22:36:15.444902 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:15.444921 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:15.444998 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:15.445193 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:15.445194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:15.445381 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:15.445430 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:15.445432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.446161 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.446162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:15.446796 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:15.446798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.447164 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.447165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:15.447204 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:15.447690 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:15.447695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:15.449568 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
W20260807 22:36:15.449767 9763 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:15.444865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:38525 (local address 127.28.82.254:46879)
0807 22:36:15.445061 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:15.445063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:15.445072 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:36:15.445095 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:15.445095 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:15.445106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:15.445171 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:15.445532 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.446079 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.446889 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.447094 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.447154 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:15.449673 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:38525: BlockingRecv error: recv got EOF from 127.28.82.193:38525 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
Aug 07 22:36:15 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:15 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:15.565590 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:15.560604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:35559)
0807 22:36:15.560824 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:15.560835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:15.560901 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:15.561026 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:15.561027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:15.561211 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:15.561257 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:15.561259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.561949 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.561950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:15.562605 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:15.562608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.562947 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.562948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:15.563008 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:15.563564 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:15.563570 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:15.565478 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":118,"threads_started":1}
W20260807 22:36:15.565629 9763 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:15.560759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:35559 (local address 127.28.82.254:46879)
0807 22:36:15.560844 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:15.560846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:15.560903 (+ 57us) server_negotiation.cc:407] Connection header received
0807 22:36:15.560965 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:15.560965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:15.560977 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:15.561021 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:15.561324 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.561870 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.562691 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.562882 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.562933 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:15.565563 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:35559: BlockingRecv error: recv got EOF from 127.28.82.194:35559 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 22:36:15 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:15 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:15.703784 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:15.698723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:45683)
0807 22:36:15.698886 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:15.698899 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:15.698995 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:15.699153 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:15.699154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:15.699351 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:15.699397 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:15.699399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.700116 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.700118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:15.700667 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:15.700670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.701036 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.701037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:15.701099 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:15.701736 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:15.701742 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:15.703724 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
W20260807 22:36:15.703969 9763 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:15.698779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:45683 (local address 127.28.82.254:46879)
0807 22:36:15.698852 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:15.698854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:15.698977 (+ 123us) server_negotiation.cc:407] Connection header received
0807 22:36:15.699053 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:15.699054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:15.699076 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:15.699143 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:15.699483 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.700020 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.700760 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:15.700957 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:15.701036 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:15.703871 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:45683: BlockingRecv error: recv got EOF from 127.28.82.195:45683 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 22:36:16 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:16 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:16.458606 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:16.450198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:59989)
0807 22:36:16.450370 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:16.450381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:16.450439 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:16.450705 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:16.450706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:16.450888 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:16.450939 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:16.450940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.451757 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.451758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:16.452382 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:16.452386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.452847 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.452848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:16.452907 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:16.456635 (+ 3728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:16.456640 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:16.458507 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
W20260807 22:36:16.458654 9768 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:16.450311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:59989 (local address 127.28.82.254:46879)
0807 22:36:16.450457 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:16.450459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:16.450534 (+ 75us) server_negotiation.cc:407] Connection header received
0807 22:36:16.450560 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:16.450561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:16.450574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:16.450692 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:16.451007 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.451627 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.452488 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.452765 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.452846 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:16.458580 (+ 5734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:59989: BlockingRecv error: recv got EOF from 127.28.82.193:59989 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
Aug 07 22:36:16 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:16 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:16.570945 9769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:16.566072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:59917)
0807 22:36:16.566251 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:16.566262 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:16.566331 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:16.566531 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:16.566532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:16.566734 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:16.566782 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:16.566784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.567501 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.567503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:16.568054 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:16.568056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.568441 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.568442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:16.568508 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:16.568975 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:16.568980 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:16.570884 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
W20260807 22:36:16.571108 9768 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:16.566293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:59917 (local address 127.28.82.254:46879)
0807 22:36:16.566370 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:16.566372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:16.566381 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:36:16.566417 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:16.566418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:16.566429 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:16.566522 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:16.566868 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.567403 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.568143 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.568351 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.568441 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:16.571022 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:59917: BlockingRecv error: recv got EOF from 127.28.82.194:59917 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 22:36:16 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:16 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:16.709885 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:16.704309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:46451)
0807 22:36:16.704488 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:16.704506 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:16.704588 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:16.704829 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:16.704830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:16.705090 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:16.705169 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:16.705171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.705888 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.705890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:16.706681 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:16.706684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.707102 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.707103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:16.707167 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:16.707682 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:16.707687 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:16.709778 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
W20260807 22:36:16.709941 9768 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:16.704357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:46451 (local address 127.28.82.254:46879)
0807 22:36:16.704448 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:16.704451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:16.704568 (+ 117us) server_negotiation.cc:407] Connection header received
0807 22:36:16.704687 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:16.704688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:16.704702 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:16.704816 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:16.705240 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.705795 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.706773 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:16.706986 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:16.707086 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:16.709870 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:46451: BlockingRecv error: recv got EOF from 127.28.82.195:46451 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 22:36:17 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:17 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:17.464159 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:17.459128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:57599)
0807 22:36:17.459280 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:17.459291 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:17.459346 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:17.459527 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:17.459528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:17.459725 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:17.459773 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:17.459775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.460474 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.460476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:17.461026 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:17.461028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.461391 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.461392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:17.461434 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:17.461995 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:17.462000 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:17.464096 (+ 2096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
W20260807 22:36:17.464282 9773 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:17.459239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:57599 (local address 127.28.82.254:46879)
0807 22:36:17.459403 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:17.459406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:17.459414 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:36:17.459436 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:17.459437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:17.459447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:17.459507 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:17.459853 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.460369 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.461119 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.461326 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.461377 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:17.464195 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:57599: BlockingRecv error: recv got EOF from 127.28.82.193:57599 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
Aug 07 22:36:17 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:17 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:17.576912 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:17.571425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:59475)
0807 22:36:17.571681 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:17.571694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:17.571794 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:17.571985 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:17.571987 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:17.572204 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:17.572252 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:17.572254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.573048 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.573049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:17.573985 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:17.573989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.574406 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.574407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:17.574486 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:17.575007 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:17.575013 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:17.576818 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":114,"threads_started":1}
W20260807 22:36:17.576977 9773 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:17.571528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:59475 (local address 127.28.82.254:46879)
0807 22:36:17.571674 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:17.571677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:17.571775 (+ 98us) server_negotiation.cc:407] Connection header received
0807 22:36:17.571895 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:17.571895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:17.571907 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:17.571964 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:17.572344 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.572860 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.574102 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.574305 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.574404 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:17.576914 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:59475: BlockingRecv error: recv got EOF from 127.28.82.194:59475 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 07 22:36:17 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:17 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.195@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:17.715480 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:17.710428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.195:55613)
0807 22:36:17.710630 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:17.710673 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:17.710732 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:17.710866 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:17.710867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:17.711035 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:17.711082 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:17.711083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.711728 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.711729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:17.712255 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:17.712256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.712605 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.712606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:17.712676 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:17.713182 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:17.713195 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:17.715410 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":134,"threads_started":1}
W20260807 22:36:17.715551 9773 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:17.710541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:55613 (local address 127.28.82.254:46879)
0807 22:36:17.710606 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:17.710609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:17.710735 (+ 126us) server_negotiation.cc:407] Connection header received
0807 22:36:17.710777 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:17.710777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:17.710790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:17.710844 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:17.711145 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.711638 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.712347 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:17.712534 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:17.712590 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:17.715501 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.195:55613: BlockingRecv error: recv got EOF from 127.28.82.195:55613 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 22:36:18 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:18 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.193@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:18.469475 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:18.464850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.193:57249)
0807 22:36:18.464983 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:18.465003 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:18.465087 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:18.465247 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:18.465248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:18.465412 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:18.465463 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:18.465465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.466203 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.466204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:18.466771 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:18.466773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.467104 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.467105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:18.467142 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:18.467637 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:18.467642 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:18.469404 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
W20260807 22:36:18.469583 9779 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:18.464916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:57249 (local address 127.28.82.254:46879)
0807 22:36:18.465067 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:18.465069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:18.465078 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:36:18.465142 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:18.465143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:18.465154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:18.465220 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:18.465548 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.466132 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.466843 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.467037 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.467093 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:18.469505 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.193:57249: BlockingRecv error: recv got EOF from 127.28.82.193:57249 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
Aug 07 22:36:18 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 22:36:18 dist-test-slave-7kzw krb5kdc[8236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.82.194@KRBTEST.COM for oryx/127.28.82.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 22:36:18.582839 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:18.577508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46879 (local address 127.28.82.194:38409)
0807 22:36:18.577708 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:18.577720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:18.577797 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:18.577963 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:18.577964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:18.578179 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:18.578229 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:18.578231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.579151 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.579152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:18.579842 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:18.579846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.580232 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.580233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:18.580289 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:18.580818 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:18.580823 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:18.582727 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.82.254:46879: Server oryx/127.28.82.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":102,"threads_started":1}
W20260807 22:36:18.582890 9779 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:18.577607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:38409 (local address 127.28.82.254:46879)
0807 22:36:18.577696 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:18.577699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:18.577772 (+ 73us) server_negotiation.cc:407] Connection header received
0807 22:36:18.577866 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:18.577867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:18.577879 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:18.577955 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:18.578332 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.578983 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.579932 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.580138 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.580233 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:18.582818 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.82.194:38409: BlockingRecv error: recv got EOF from 127.28.82.194:38409 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 22:36:18.623399 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 9082
I20260807 22:36:18.636476 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 9311
I20260807 22:36:18.645092 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 9496
I20260807 22:36:18.658244 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 8978
2026-08-07T22:36:18Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17745 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T22:36:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:36:18Z Disabled control of system clock
I20260807 22:36:18.690310 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46713
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 22:36:18.774204 9784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:18.774513 9784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:18.774540 9784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:18.775687 9784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:36:18.775727 9784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:18.775739 9784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:18.775753 9784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:36:18.775771 9784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:36:18.777156 9784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:46713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:18.777369 9784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:18.777606 9784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:18.783001 9793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:18.783036 9792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:18.783046 9795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:18.783063 9784 server_base.cc:1034] running on GCE node
I20260807 22:36:18.783453 9784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:18.783751 9784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:18.784924 9784 hybrid_clock.cc:648] HybridClock initialized: now 1786142178784903 us; error 37 us; skew 500 ppm
I20260807 22:36:18.786296 9784 webserver.cc:460] Webserver started at http://127.28.82.254:33555/ using document root <none> and password file <none>
I20260807 22:36:18.786595 9784 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:18.786676 9784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:18.786840 9784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:18.787907 9784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "44cc17d7e70c4e129f6445123fbd77cb"
format_stamp: "Formatted at 2026-08-07 22:36:18 on dist-test-slave-7kzw"
server_key: "225cb749a216242567ac6081bfce037a"
server_key_iv: "bf41067fddc7fc5f1fa3e5acd6fddf00"
server_key_version: "encryptionkey@0"
I20260807 22:36:18.788290 9784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "44cc17d7e70c4e129f6445123fbd77cb"
format_stamp: "Formatted at 2026-08-07 22:36:18 on dist-test-slave-7kzw"
server_key: "225cb749a216242567ac6081bfce037a"
server_key_iv: "bf41067fddc7fc5f1fa3e5acd6fddf00"
server_key_version: "encryptionkey@0"
I20260807 22:36:18.789726 9784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:36:18.790771 9805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:18.791021 9784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:36:18.791138 9784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "44cc17d7e70c4e129f6445123fbd77cb"
format_stamp: "Formatted at 2026-08-07 22:36:18 on dist-test-slave-7kzw"
server_key: "225cb749a216242567ac6081bfce037a"
server_key_iv: "bf41067fddc7fc5f1fa3e5acd6fddf00"
server_key_version: "encryptionkey@0"
I20260807 22:36:18.791227 9784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:18.801201 9784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:18.801534 9784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:18.801710 9784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:18.806301 9784 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:46713
I20260807 22:36:18.806354 9889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:46713 every 8 connection(s)
I20260807 22:36:18.806785 9784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:36:18.807502 9890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:36:18.810302 9890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb: Bootstrap starting.
I20260807 22:36:18.810967 9890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb: Neither blocks nor log segments found. Creating new log.
I20260807 22:36:18.811290 9890 log.cc:824] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb: Log is configured to *not* fsync() on all Append() calls
I20260807 22:36:18.812058 9890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb: No bootstrap required, opened a new log
I20260807 22:36:18.813657 9890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cc17d7e70c4e129f6445123fbd77cb" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46713 } }
I20260807 22:36:18.813807 9890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:36:18.813830 9890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44cc17d7e70c4e129f6445123fbd77cb, State: Initialized, Role: FOLLOWER
I20260807 22:36:18.813988 9890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cc17d7e70c4e129f6445123fbd77cb" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46713 } }
I20260807 22:36:18.814065 9890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:36:18.814109 9890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:36:18.814159 9890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:36:18.814802 9890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cc17d7e70c4e129f6445123fbd77cb" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46713 } }
I20260807 22:36:18.814898 9890 leader_election.cc:302] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44cc17d7e70c4e129f6445123fbd77cb; no voters:
I20260807 22:36:18.815076 9890 leader_election.cc:288] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:36:18.815126 9893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:36:18.815224 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 9784
I20260807 22:36:18.815285 9893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [term 1 LEADER]: Becoming Leader. State: Replica: 44cc17d7e70c4e129f6445123fbd77cb, State: Running, Role: LEADER
I20260807 22:36:18.815340 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:36:18.815374 9893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cc17d7e70c4e129f6445123fbd77cb" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46713 } }
I20260807 22:36:18.815373 9890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:36:18.815594 29003 external_mini_cluster.cc:1428] Setting key 08769d63883c0e0f4d864aab95e42950
I20260807 22:36:18.816013 9896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44cc17d7e70c4e129f6445123fbd77cb. Latest consensus state: current_term: 1 leader_uuid: "44cc17d7e70c4e129f6445123fbd77cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cc17d7e70c4e129f6445123fbd77cb" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46713 } } }
I20260807 22:36:18.816114 9896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:18.816376 9895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44cc17d7e70c4e129f6445123fbd77cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44cc17d7e70c4e129f6445123fbd77cb" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46713 } } }
I20260807 22:36:18.816506 9895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:18.816689 9909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:36:18.817255 9909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:36:18.819895 9909 catalog_manager.cc:1347] Generated new cluster ID: 39ba406d0c1f4a71b798cdee246597b2
I20260807 22:36:18.819958 9909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:36:18.821944 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:18.816308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.28.82.254:46713)
0807 22:36:18.816529 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:18.816538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:18.816787 (+ 249us) server_negotiation.cc:407] Connection header received
0807 22:36:18.816920 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:18.816930 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:18.817009 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:18.817154 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:18.817432 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.818726 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.819831 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.820309 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.820448 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:18.821250 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:18.821253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:18.821254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:36:18.821440 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:18.821473 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:36:18.821484 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:18.821560 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:18.821645 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 22:36:18.821730 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260807 22:36:18.835134 9909 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:36:18.835842 9909 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:36:18.841053 9909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44cc17d7e70c4e129f6445123fbd77cb: Generated new TSK 0
I20260807 22:36:18.841238 9909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:36:18.845959 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46713
--builtin_ntp_servers=127.28.82.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 22:36:18.932010 9923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:18.932263 9923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:18.932300 9923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:18.933445 9923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:18.933513 9923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:18.933557 9923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:36:18.935005 9923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:18.935279 9923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:18.935598 9923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:18.941937 9930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:18.941921 9933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:18.941926 9935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:18.942381 9923 server_base.cc:1034] running on GCE node
I20260807 22:36:18.942629 9923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:18.942925 9923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:18.944108 9923 hybrid_clock.cc:648] HybridClock initialized: now 1786142178944089 us; error 33 us; skew 500 ppm
I20260807 22:36:18.945560 9923 webserver.cc:460] Webserver started at http://127.28.82.193:42543/ using document root <none> and password file <none>
I20260807 22:36:18.945827 9923 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:18.945883 9923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:18.946002 9923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:18.947425 9923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "8454459a975e4f018edb58fa75c1e3a2"
format_stamp: "Formatted at 2026-08-07 22:36:18 on dist-test-slave-7kzw"
server_key: "8ab2a0ecbdffef4ebbc4b0228790d772"
server_key_iv: "714c8bb95a99662317d3250481f3f5d2"
server_key_version: "encryptionkey@0"
I20260807 22:36:18.947831 9923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "8454459a975e4f018edb58fa75c1e3a2"
format_stamp: "Formatted at 2026-08-07 22:36:18 on dist-test-slave-7kzw"
server_key: "8ab2a0ecbdffef4ebbc4b0228790d772"
server_key_iv: "714c8bb95a99662317d3250481f3f5d2"
server_key_version: "encryptionkey@0"
I20260807 22:36:18.949473 9923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:36:18.950572 9944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:18.950785 9923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:18.950881 9923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "8454459a975e4f018edb58fa75c1e3a2"
format_stamp: "Formatted at 2026-08-07 22:36:18 on dist-test-slave-7kzw"
server_key: "8ab2a0ecbdffef4ebbc4b0228790d772"
server_key_iv: "714c8bb95a99662317d3250481f3f5d2"
server_key_version: "encryptionkey@0"
I20260807 22:36:18.950953 9923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:18.965500 9923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:18.965767 9923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:18.965871 9923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:18.966048 9923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:18.966328 9923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:18.966367 9923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:18.966421 9923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:18.966442 9923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:18.973699 9923 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:40651
I20260807 22:36:18.973747 10108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:40651 every 8 connection(s)
I20260807 22:36:18.974128 9923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:36:18.979911 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:18.974921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:54675 (local address 127.28.82.254:46713)
0807 22:36:18.974989 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:18.974992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:18.975290 (+ 298us) server_negotiation.cc:407] Connection header received
0807 22:36:18.975904 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:18.975906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:18.975919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:18.975979 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:18.976731 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.977250 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.978351 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.978669 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.978741 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:18.979600 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:18.979602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:18.979602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:36:18.979608 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:18.979627 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:36:18.979632 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:18.979698 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:18.979812 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 22:36:18.979845 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:36:18.980008 10113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:18.974911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46713 (local address 127.28.82.193:54675)
0807 22:36:18.975159 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:18.975198 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:18.975778 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:18.976034 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:18.976037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:18.976046 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:18.976623 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:18.976629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.977341 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.977342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:18.978249 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:18.978252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:18.978761 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:18.978762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:18.978887 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:18.979431 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:36:18.979432 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:18.979460 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:36:18.979482 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:18.979700 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:18.979703 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:18.979706 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:36:18.979759 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 22:36:18.979844 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
I20260807 22:36:18.980621 10109 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46713
I20260807 22:36:18.980748 10109 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:18.981005 10109 heartbeater.cc:511] Master 127.28.82.254:46713 requested a full tablet report, sending...
I20260807 22:36:18.981652 9832 ts_manager.cc:194] Registered new tserver with Master: 8454459a975e4f018edb58fa75c1e3a2 (127.28.82.193:40651)
I20260807 22:36:18.982389 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 9923
I20260807 22:36:18.982420 9832 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.193:54675
I20260807 22:36:18.982550 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:36:18.982833 29003 external_mini_cluster.cc:1428] Setting key a0988ac697d5c56491ee9a08adbafd58
I20260807 22:36:18.983884 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46713
--builtin_ntp_servers=127.28.82.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 22:36:19.069962 10116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:19.070210 10116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:19.070236 10116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:19.071496 10116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:19.071552 10116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:19.071581 10116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:36:19.072937 10116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:19.073203 10116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:19.073542 10116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:19.078933 10130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.078919 10127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.079020 10128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:19.079130 10116 server_base.cc:1034] running on GCE node
I20260807 22:36:19.079412 10116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:19.079628 10116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:19.080780 10116 hybrid_clock.cc:648] HybridClock initialized: now 1786142179080766 us; error 29 us; skew 500 ppm
I20260807 22:36:19.082122 10116 webserver.cc:460] Webserver started at http://127.28.82.194:41547/ using document root <none> and password file <none>
I20260807 22:36:19.082382 10116 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:19.082454 10116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:19.082638 10116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:19.083719 10116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "5288e587f7f5418badefef8cb3f80b43"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "fcb4e4f1ddb5888bcd5bac0b4c320998"
server_key_iv: "4c20c398d261ed6443864297d53e990e"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.084097 10116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "5288e587f7f5418badefef8cb3f80b43"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "fcb4e4f1ddb5888bcd5bac0b4c320998"
server_key_iv: "4c20c398d261ed6443864297d53e990e"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.085464 10116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:36:19.086292 10137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.086530 10116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:19.086642 10116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "5288e587f7f5418badefef8cb3f80b43"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "fcb4e4f1ddb5888bcd5bac0b4c320998"
server_key_iv: "4c20c398d261ed6443864297d53e990e"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.086731 10116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:19.110559 10116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:19.110919 10116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:19.111081 10116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:19.111359 10116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:19.111743 10116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:19.111809 10116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.111846 10116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:19.111889 10116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.118113 10116 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:35657
I20260807 22:36:19.118199 10297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:35657 every 8 connection(s)
I20260807 22:36:19.118606 10116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:36:19.119892 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 10116
I20260807 22:36:19.119999 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:36:19.120206 29003 external_mini_cluster.cc:1428] Setting key d69ecedbf79fa2a1e7718621661823b2
I20260807 22:36:19.121220 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46713
--builtin_ntp_servers=127.28.82.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 22:36:19.124465 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.119388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:48759 (local address 127.28.82.254:46713)
0807 22:36:19.119454 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:19.119456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:19.119702 (+ 246us) server_negotiation.cc:407] Connection header received
0807 22:36:19.120323 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:19.120325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:19.120338 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:19.120402 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:19.121309 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:19.121812 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:19.122902 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:19.123169 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:19.123233 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:19.124191 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:19.124192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:19.124193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:36:19.124197 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:19.124212 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:36:19.124216 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:19.124263 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:19.124380 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 22:36:19.124406 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:36:19.124742 10305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.119372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46713 (local address 127.28.82.194:48759)
0807 22:36:19.119577 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:19.119611 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:19.120206 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:19.120595 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:19.120599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:19.120607 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:19.121210 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:19.121215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:19.121912 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:19.121913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:19.122802 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:19.122806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:19.123262 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:19.123263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:19.123428 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:19.124022 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:36:19.124023 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:19.124071 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:36:19.124100 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:19.124292 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:19.124298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:19.124301 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:36:19.124375 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:36:19.124486 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260807 22:36:19.125237 10303 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46713
I20260807 22:36:19.125349 10303 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:19.125535 10303 heartbeater.cc:511] Master 127.28.82.254:46713 requested a full tablet report, sending...
I20260807 22:36:19.125994 9832 ts_manager.cc:194] Registered new tserver with Master: 5288e587f7f5418badefef8cb3f80b43 (127.28.82.194:35657)
I20260807 22:36:19.126380 9832 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.194:48759
W20260807 22:36:19.205487 10307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:19.205747 10307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:19.205785 10307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:19.207015 10307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:19.207077 10307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:19.207114 10307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:36:19.208451 10307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:19.208727 10307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:19.209038 10307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:19.214257 10319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.214286 10317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:19.214402 10307 server_base.cc:1034] running on GCE node
W20260807 22:36:19.214363 10316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:19.214705 10307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:19.214972 10307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:19.216178 10307 hybrid_clock.cc:648] HybridClock initialized: now 1786142179216160 us; error 28 us; skew 500 ppm
I20260807 22:36:19.217780 10307 webserver.cc:460] Webserver started at http://127.28.82.195:34301/ using document root <none> and password file <none>
I20260807 22:36:19.218045 10307 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:19.218127 10307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:19.218297 10307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:19.219523 10307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "14f6236bc3964f8eb3df34b17f841f06"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "fe00a705c487c75245f6c8705d5d8421"
server_key_iv: "9a26e73ac1db0c19dbd408a435dcdbc0"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.219949 10307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "14f6236bc3964f8eb3df34b17f841f06"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "fe00a705c487c75245f6c8705d5d8421"
server_key_iv: "9a26e73ac1db0c19dbd408a435dcdbc0"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.221393 10307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:36:19.222180 10326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.222353 10307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:36:19.222461 10307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "14f6236bc3964f8eb3df34b17f841f06"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "fe00a705c487c75245f6c8705d5d8421"
server_key_iv: "9a26e73ac1db0c19dbd408a435dcdbc0"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.222584 10307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:19.244299 10307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:19.244639 10307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:19.244802 10307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:19.245054 10307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:19.245416 10307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:19.245474 10307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.245517 10307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:19.245545 10307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.251580 10307 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:41007
I20260807 22:36:19.251709 10483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:41007 every 8 connection(s)
I20260807 22:36:19.252007 10307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:36:19.257522 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.252784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:39165 (local address 127.28.82.254:46713)
0807 22:36:19.252837 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:19.252839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:19.253104 (+ 265us) server_negotiation.cc:407] Connection header received
0807 22:36:19.253560 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:19.253562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:19.253572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:19.253675 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:19.254333 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:19.255052 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:19.256105 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:19.256296 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:19.256357 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:19.257217 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:19.257218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:19.257219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:36:19.257223 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:19.257250 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:36:19.257254 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:19.257320 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:19.257433 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 22:36:19.257459 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:36:19.257740 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.252751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46713 (local address 127.28.82.195:39165)
0807 22:36:19.252960 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:19.253002 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:19.253475 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:19.253730 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:19.253733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:19.253741 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:19.254213 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:19.254217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:19.255152 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:19.255153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:19.256012 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:19.256015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:19.256373 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:19.256374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:19.256481 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:19.257044 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:36:19.257044 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:19.257073 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:36:19.257096 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:19.257339 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:19.257344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:19.257347 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:36:19.257407 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 22:36:19.257492 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260807 22:36:19.258034 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 10307
I20260807 22:36:19.258147 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:36:19.258306 10484 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46713
I20260807 22:36:19.258378 29003 external_mini_cluster.cc:1428] Setting key d42a8d2feeaded786fdce25a7777ae0b
I20260807 22:36:19.258430 10484 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:19.258703 10484 heartbeater.cc:511] Master 127.28.82.254:46713 requested a full tablet report, sending...
I20260807 22:36:19.259109 9832 ts_manager.cc:194] Registered new tserver with Master: 14f6236bc3964f8eb3df34b17f841f06 (127.28.82.195:41007)
I20260807 22:36:19.259517 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:36:19.259595 9832 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.195:39165
W20260807 22:36:19.261467 9912 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:19.260935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.28.82.254:46713)
0807 22:36:19.261011 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:19.261014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:19.261110 (+ 96us) server_negotiation.cc:407] Connection header received
0807 22:36:19.261169 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:19.261170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:19.261182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:19.261253 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:19.261389 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43742: BlockingRecv error: recv got EOF from 127.0.0.1:43742 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:36:19.261505 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 9923
I20260807 22:36:19.271411 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 10116
I20260807 22:36:19.283707 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 10307
I20260807 22:36:19.299044 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 9784
2026-08-07T22:36:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (641 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T22:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:36:19Z Disabled control of system clock
I20260807 22:36:19.331437 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37195
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:45479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:37195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:36:19.413323 10504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:19.413532 10504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:19.413553 10504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:19.414752 10504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:36:19.414794 10504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:19.414808 10504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:19.414820 10504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:36:19.414832 10504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:36:19.416253 10504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:37195
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:19.416517 10504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:19.416811 10504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:19.422010 10520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.421950 10519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.421973 10522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:19.422400 10504 server_base.cc:1034] running on GCE node
I20260807 22:36:19.422780 10504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:19.423085 10504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:19.424253 10504 hybrid_clock.cc:648] HybridClock initialized: now 1786142179424228 us; error 42 us; skew 500 ppm
I20260807 22:36:19.425578 10504 webserver.cc:460] Webserver started at http://127.28.82.254:36999/ using document root <none> and password file <none>
I20260807 22:36:19.425838 10504 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:19.425912 10504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:19.426079 10504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:19.427227 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "1b4761fe01a14dc995758e7a2877d283"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "bc9b1aeda066bf8f705197a2450ce5b3"
server_key_iv: "30f0dc85f03770e67cb7ce89b207f504"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.427639 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "1b4761fe01a14dc995758e7a2877d283"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "bc9b1aeda066bf8f705197a2450ce5b3"
server_key_iv: "30f0dc85f03770e67cb7ce89b207f504"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.428990 10504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:36:19.429920 10531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.430138 10504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:19.430253 10504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "1b4761fe01a14dc995758e7a2877d283"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "bc9b1aeda066bf8f705197a2450ce5b3"
server_key_iv: "30f0dc85f03770e67cb7ce89b207f504"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.430332 10504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:19.433828 10504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:19.434135 10504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:19.434285 10504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:19.438370 10504 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:37195
I20260807 22:36:19.438403 10600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:37195 every 8 connection(s)
I20260807 22:36:19.438748 10504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:36:19.439411 10601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:36:19.442145 10601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283: Bootstrap starting.
I20260807 22:36:19.442754 10601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283: Neither blocks nor log segments found. Creating new log.
I20260807 22:36:19.443020 10601 log.cc:824] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283: Log is configured to *not* fsync() on all Append() calls
I20260807 22:36:19.443703 10601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283: No bootstrap required, opened a new log
I20260807 22:36:19.445240 10601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4761fe01a14dc995758e7a2877d283" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37195 } }
I20260807 22:36:19.445375 10601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:36:19.445398 10601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4761fe01a14dc995758e7a2877d283, State: Initialized, Role: FOLLOWER
I20260807 22:36:19.445545 10601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4761fe01a14dc995758e7a2877d283" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37195 } }
I20260807 22:36:19.445649 10601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:36:19.445675 10601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:36:19.445705 10601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:36:19.445998 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 10504
I20260807 22:36:19.446111 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:36:19.446333 29003 external_mini_cluster.cc:1428] Setting key 96b130c78a4c95a55a7bbd886f26cf99
I20260807 22:36:19.446513 10601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4761fe01a14dc995758e7a2877d283" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37195 } }
I20260807 22:36:19.446630 10601 leader_election.cc:302] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b4761fe01a14dc995758e7a2877d283; no voters:
I20260807 22:36:19.446815 10601 leader_election.cc:288] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:36:19.446849 10605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:36:19.447115 10605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [term 1 LEADER]: Becoming Leader. State: Replica: 1b4761fe01a14dc995758e7a2877d283, State: Running, Role: LEADER
I20260807 22:36:19.447194 10601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:36:19.447294 10605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4761fe01a14dc995758e7a2877d283" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37195 } }
I20260807 22:36:19.447955 10608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b4761fe01a14dc995758e7a2877d283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4761fe01a14dc995758e7a2877d283" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37195 } } }
I20260807 22:36:19.448050 10608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:19.448295 10609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b4761fe01a14dc995758e7a2877d283. Latest consensus state: current_term: 1 leader_uuid: "1b4761fe01a14dc995758e7a2877d283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b4761fe01a14dc995758e7a2877d283" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37195 } } }
I20260807 22:36:19.448421 10609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283 [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:19.448913 10618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:36:19.448925 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.446882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.28.82.254:37195)
0807 22:36:19.447253 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:19.447263 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:19.447316 (+ 53us) server_negotiation.cc:407] Connection header received
0807 22:36:19.447437 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:19.447446 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:19.447494 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:19.447635 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:19.448391 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:19.448395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:19.448396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:36:19.448400 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:19.448450 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:36:19.448466 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:19.448542 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:19.448659 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 22:36:19.448721 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":78,"threads_started":1}
I20260807 22:36:19.449491 10618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:36:19.451067 10618 catalog_manager.cc:1347] Generated new cluster ID: acc54d343b3f40909666a9530d4a3a78
I20260807 22:36:19.451131 10618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:36:19.488845 10618 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:36:19.489053 10618 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:36:19.492331 10618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b4761fe01a14dc995758e7a2877d283: Generated new TSK 0
I20260807 22:36:19.492522 10618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:36:19.504199 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37195
--builtin_ntp_servers=127.28.82.212:45479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:36:19.589011 10632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:19.589253 10632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:19.589277 10632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:19.590617 10632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:19.590664 10632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:19.590692 10632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:36:19.592051 10632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:19.592250 10632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:19.592502 10632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:19.597962 10642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.597966 10640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.597971 10639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:19.598443 10632 server_base.cc:1034] running on GCE node
I20260807 22:36:19.598788 10632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:19.599215 10632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:19.600417 10632 hybrid_clock.cc:648] HybridClock initialized: now 1786142179600416 us; error 46 us; skew 500 ppm
I20260807 22:36:19.601789 10632 webserver.cc:460] Webserver started at http://127.28.82.193:41765/ using document root <none> and password file <none>
I20260807 22:36:19.602048 10632 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:19.602121 10632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:19.602280 10632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:19.603443 10632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "06670e024dfb4e96b5afb41161089a9c"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "23ebd97e60d84b582b7d811f77e69e90"
server_key_iv: "601703259e2921b9124f6937e090f858"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.603832 10632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "06670e024dfb4e96b5afb41161089a9c"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "23ebd97e60d84b582b7d811f77e69e90"
server_key_iv: "601703259e2921b9124f6937e090f858"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.605271 10632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:36:19.606110 10650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.606295 10632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:19.606407 10632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "06670e024dfb4e96b5afb41161089a9c"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "23ebd97e60d84b582b7d811f77e69e90"
server_key_iv: "601703259e2921b9124f6937e090f858"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.606519 10632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:19.609927 10632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:19.610250 10632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:19.610401 10632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:19.610638 10632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:19.610961 10632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:19.611017 10632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.611059 10632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:19.611086 10632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.616770 10632 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:45299
I20260807 22:36:19.616808 10810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:45299 every 8 connection(s)
I20260807 22:36:19.617161 10632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:36:19.620054 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 10632
I20260807 22:36:19.620164 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:36:19.620206 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.617949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:39845 (local address 127.28.82.254:37195)
0807 22:36:19.617994 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:19.617996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:19.618264 (+ 268us) server_negotiation.cc:407] Connection header received
0807 22:36:19.618771 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:19.618773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:19.618780 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:19.618834 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:19.619918 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:19.619920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:19.619921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:36:19.619927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:19.619947 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:36:19.619952 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:19.620028 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:19.620087 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 22:36:19.620104 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:36:19.620378 29003 external_mini_cluster.cc:1428] Setting key 09c1f3544af261720157ab355dccb4ba
I20260807 22:36:19.620397 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.617922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37195 (local address 127.28.82.193:39845)
0807 22:36:19.618133 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:19.618166 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:19.618675 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:19.618883 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:19.618886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:19.618894 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:19.619769 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:36:19.619770 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:19.619797 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:36:19.619824 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:19.620024 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:19.620026 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:19.620028 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:36:19.620078 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 22:36:19.620148 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":124,"threads_started":1}
I20260807 22:36:19.620990 10812 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37195
I20260807 22:36:19.621090 10812 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:19.621243 10812 heartbeater.cc:511] Master 127.28.82.254:37195 requested a full tablet report, sending...
I20260807 22:36:19.621447 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37195
--builtin_ntp_servers=127.28.82.212:45479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 22:36:19.621783 10557 ts_manager.cc:194] Registered new tserver with Master: 06670e024dfb4e96b5afb41161089a9c (127.28.82.193:45299)
W20260807 22:36:19.709939 10816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:19.710203 10816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:19.710227 10816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:19.711414 10816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:19.711459 10816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:19.711484 10816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:36:19.712786 10816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:19.712978 10816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:19.713227 10816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:19.718278 10826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.718267 10824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.718263 10823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:19.719071 10816 server_base.cc:1034] running on GCE node
I20260807 22:36:19.719295 10816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:19.719529 10816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:19.720707 10816 hybrid_clock.cc:648] HybridClock initialized: now 1786142179720683 us; error 54 us; skew 500 ppm
I20260807 22:36:19.721984 10816 webserver.cc:460] Webserver started at http://127.28.82.194:33681/ using document root <none> and password file <none>
I20260807 22:36:19.722242 10816 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:19.722316 10816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:19.722509 10816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:19.723573 10816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "b0758afaf3e04e6c9a4918a3a730cac7"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "64d8cd017787689324e184ccde9a9b8f"
server_key_iv: "80f28738f29f8259d8c29de747fcdbc1"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.723945 10816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "b0758afaf3e04e6c9a4918a3a730cac7"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "64d8cd017787689324e184ccde9a9b8f"
server_key_iv: "80f28738f29f8259d8c29de747fcdbc1"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.725282 10816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:36:19.726055 10833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.726274 10816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:36:19.726385 10816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "b0758afaf3e04e6c9a4918a3a730cac7"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "64d8cd017787689324e184ccde9a9b8f"
server_key_iv: "80f28738f29f8259d8c29de747fcdbc1"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.726459 10816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:19.729890 10816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:19.730156 10816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:19.730304 10816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:19.730623 10816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:19.730938 10816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:19.730993 10816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.731032 10816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:19.731060 10816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.737340 10816 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:37793
I20260807 22:36:19.737376 10997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:37793 every 8 connection(s)
I20260807 22:36:19.737738 10816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:36:19.740916 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.738552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:50715 (local address 127.28.82.254:37195)
0807 22:36:19.738623 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:19.738626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:19.738831 (+ 205us) server_negotiation.cc:407] Connection header received
0807 22:36:19.739327 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:19.739329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:19.739336 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:19.739388 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:19.740617 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:19.740619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:19.740619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:36:19.740625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:19.740644 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:36:19.740648 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:19.740703 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:19.740821 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 22:36:19.740837 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:36:19.741108 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.738520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37195 (local address 127.28.82.194:50715)
0807 22:36:19.738718 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:19.738750 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:19.739210 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:19.739453 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:19.739457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:19.739466 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:19.740461 (+ 995us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:36:19.740461 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:19.740489 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:36:19.740516 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:19.740724 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:19.740729 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:19.740732 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:36:19.740831 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 22:36:19.740896 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260807 22:36:19.741616 10998 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37195
I20260807 22:36:19.741698 10998 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:19.741885 10998 heartbeater.cc:511] Master 127.28.82.254:37195 requested a full tablet report, sending...
I20260807 22:36:19.742256 10557 ts_manager.cc:194] Registered new tserver with Master: b0758afaf3e04e6c9a4918a3a730cac7 (127.28.82.194:37793)
I20260807 22:36:19.747545 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 10816
I20260807 22:36:19.747632 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:36:19.747864 29003 external_mini_cluster.cc:1428] Setting key 4ef2e72b5dad42b90ecbaee6f4b0b1a5
I20260807 22:36:19.749112 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37195
--builtin_ntp_servers=127.28.82.212:45479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 22:36:19.836385 11001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:19.836642 11001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:19.836684 11001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:19.837988 11001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:19.838064 11001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:19.838114 11001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:36:19.839699 11001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:19.839993 11001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:19.840323 11001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:19.845983 11015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:19.846169 11001 server_base.cc:1034] running on GCE node
W20260807 22:36:19.846009 11013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:19.845988 11012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:19.846549 11001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:19.846818 11001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:19.847959 11001 hybrid_clock.cc:648] HybridClock initialized: now 1786142179847945 us; error 30 us; skew 500 ppm
I20260807 22:36:19.849277 11001 webserver.cc:460] Webserver started at http://127.28.82.195:37895/ using document root <none> and password file <none>
I20260807 22:36:19.849531 11001 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:19.849578 11001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:19.849776 11001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:19.850910 11001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "12e9c10be3274030850289548bdfecae"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "73188feba61eeda234bc870adc4a294e"
server_key_iv: "44fbd43f82f4afbe8b12ad8d74f24cb4"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.851274 11001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "12e9c10be3274030850289548bdfecae"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "73188feba61eeda234bc870adc4a294e"
server_key_iv: "44fbd43f82f4afbe8b12ad8d74f24cb4"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.852689 11001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 22:36:19.853518 11024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.853722 11001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:19.853813 11001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "12e9c10be3274030850289548bdfecae"
format_stamp: "Formatted at 2026-08-07 22:36:19 on dist-test-slave-7kzw"
server_key: "73188feba61eeda234bc870adc4a294e"
server_key_iv: "44fbd43f82f4afbe8b12ad8d74f24cb4"
server_key_version: "encryptionkey@0"
I20260807 22:36:19.853868 11001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:19.857177 11001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:19.857388 11001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:19.857481 11001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:19.857628 11001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:19.857889 11001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:19.857920 11001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.857941 11001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:19.857954 11001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:19.863668 11001 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:36643
I20260807 22:36:19.863790 11201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:36643 every 8 connection(s)
I20260807 22:36:19.864092 11001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:36:19.864902 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 11001
I20260807 22:36:19.865023 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:36:19.865224 29003 external_mini_cluster.cc:1428] Setting key 5932a5c18c34c7881e96ad20f6600364
I20260807 22:36:19.867507 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.864985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:40721 (local address 127.28.82.254:37195)
0807 22:36:19.865044 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:19.865047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:19.865294 (+ 247us) server_negotiation.cc:407] Connection header received
0807 22:36:19.865934 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:19.865935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:19.865940 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:19.866021 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:19.867300 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:19.867301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:19.867301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:36:19.867305 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:19.867321 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:36:19.867326 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:19.867373 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:19.867438 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 22:36:19.867452 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 22:36:19.867745 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:19.864964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37195 (local address 127.28.82.195:40721)
0807 22:36:19.865156 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:19.865196 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:19.865815 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:19.866091 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:19.866095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:19.866105 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:19.867141 (+ 1036us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:36:19.867142 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:19.867171 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:36:19.867200 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:19.867373 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:19.867376 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:19.867379 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:36:19.867431 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 22:36:19.867479 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260807 22:36:19.868327 11202 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37195
I20260807 22:36:19.868417 11202 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:19.868571 11202 heartbeater.cc:511] Master 127.28.82.254:37195 requested a full tablet report, sending...
I20260807 22:36:19.868968 10557 ts_manager.cc:194] Registered new tserver with Master: 12e9c10be3274030850289548bdfecae (127.28.82.195:36643)
I20260807 22:36:19.877113 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 22:36:19.879172 10607 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:36:19.878569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.28.82.254:37195)
0807 22:36:19.878688 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:19.878691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:19.878800 (+ 109us) server_negotiation.cc:407] Connection header received
0807 22:36:19.878896 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:19.878897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:19.878903 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:19.878950 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:19.879102 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41632: BlockingRecv error: recv got EOF from 127.0.0.1:41632 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:36:19.879274 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 10632
I20260807 22:36:19.893087 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 10816
I20260807 22:36:19.905318 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 11001
I20260807 22:36:19.917443 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 10504
2026-08-07T22:36:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (620 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:36:19 dist-test-slave-7kzw krb5kdc[11218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:36:19 dist-test-slave-7kzw krb5kdc[11218](info): set up 2 sockets
Aug 07 22:36:19 dist-test-slave-7kzw krb5kdc[11218](info): commencing operation
krb5kdc: starting...
W20260807 22:36:21.171617 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:36:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:36:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:36:21.306170 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43037
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:43037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:21.389719 11238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:21.389990 11238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:21.390033 11238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:21.391227 11238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:36:21.391310 11238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:21.391337 11238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:21.391359 11238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:36:21.391386 11238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:36:21.392850 11238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:43037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:21.393141 11238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:21.393532 11238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:21.398754 11246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:21.398754 11249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:21.398866 11247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:21.399405 11238 server_base.cc:1034] running on GCE node
I20260807 22:36:21.399631 11238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:21.399971 11238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:21.401160 11238 hybrid_clock.cc:648] HybridClock initialized: now 1786142181401130 us; error 43 us; skew 500 ppm
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:21.403934 11238 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:36:21.404395 11238 webserver.cc:460] Webserver started at http://127.28.82.254:34891/ using document root <none> and password file <none>
I20260807 22:36:21.404632 11238 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:21.404711 11238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:21.404870 11238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:21.405954 11238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "701a06dbcfff431f997d79e43641cd91"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "0323635f33844cb53be9db6926ab13bf"
server_key_iv: "1311a42c6f56193e5ee49bacae740155"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.406347 11238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "701a06dbcfff431f997d79e43641cd91"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "0323635f33844cb53be9db6926ab13bf"
server_key_iv: "1311a42c6f56193e5ee49bacae740155"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.407886 11238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:21.408885 11257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.409221 11238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:21.409363 11238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "701a06dbcfff431f997d79e43641cd91"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "0323635f33844cb53be9db6926ab13bf"
server_key_iv: "1311a42c6f56193e5ee49bacae740155"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.409456 11238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:21.419847 11238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:21.422228 11238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:21.422576 11238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:21.427000 11238 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:43037
I20260807 22:36:21.427039 11337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:43037 every 8 connection(s)
I20260807 22:36:21.427433 11238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:36:21.428458 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:36:21.430893 11338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91: Bootstrap starting.
I20260807 22:36:21.431519 11338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91: Neither blocks nor log segments found. Creating new log.
I20260807 22:36:21.431854 11338 log.cc:824] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91: Log is configured to *not* fsync() on all Append() calls
I20260807 22:36:21.432099 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 11238
I20260807 22:36:21.432242 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:36:21.432456 29003 external_mini_cluster.cc:1428] Setting key 2909497519ae669f11c3f1430c813995
I20260807 22:36:21.432627 11338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91: No bootstrap required, opened a new log
I20260807 22:36:21.434563 11338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a06dbcfff431f997d79e43641cd91" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43037 } }
I20260807 22:36:21.434726 11338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:36:21.434793 11338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701a06dbcfff431f997d79e43641cd91, State: Initialized, Role: FOLLOWER
I20260807 22:36:21.434952 11338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a06dbcfff431f997d79e43641cd91" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43037 } }
I20260807 22:36:21.435061 11338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:36:21.435109 11338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:36:21.435168 11338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:36:21.435956 11338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a06dbcfff431f997d79e43641cd91" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43037 } }
I20260807 22:36:21.436087 11338 leader_election.cc:302] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 701a06dbcfff431f997d79e43641cd91; no voters:
I20260807 22:36:21.436275 11338 leader_election.cc:288] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:36:21.436319 11346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:36:21.436439 11346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [term 1 LEADER]: Becoming Leader. State: Replica: 701a06dbcfff431f997d79e43641cd91, State: Running, Role: LEADER
I20260807 22:36:21.436548 11338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:36:21.436586 11346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a06dbcfff431f997d79e43641cd91" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43037 } }
I20260807 22:36:21.437052 11348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 701a06dbcfff431f997d79e43641cd91. Latest consensus state: current_term: 1 leader_uuid: "701a06dbcfff431f997d79e43641cd91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a06dbcfff431f997d79e43641cd91" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43037 } } }
I20260807 22:36:21.437188 11348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:21.437057 11347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "701a06dbcfff431f997d79e43641cd91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a06dbcfff431f997d79e43641cd91" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43037 } } }
I20260807 22:36:21.437285 11347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91 [sys.catalog]: This master's current role is: LEADER
I20260807 22:36:21.438246 11357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:36:21.438997 11357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:36:21.440627 11357 catalog_manager.cc:1347] Generated new cluster ID: fa6dcba514b54612bf851ff82d9b18d3
I20260807 22:36:21.440682 11357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:36:21.445592 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.433031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.28.82.254:43037)
0807 22:36:21.433230 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:21.433239 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:21.433273 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:36:21.434553 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:21.434567 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:21.434635 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:21.434796 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:21.435528 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.436644 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.437693 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.438356 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.438495 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.441482 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:21.441486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:21.441486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:21.441498 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:21.443612 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.444019 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.444022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.444023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.444168 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.444448 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.444449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.444450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.444782 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:21.444956 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:21.445054 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 22:36:21.445199 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":108,"threads_started":1}
I20260807 22:36:21.445770 11357 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:36:21.446534 11357 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:36:21.451877 11357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91: Generated new TSK 0
I20260807 22:36:21.452049 11357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:36:21.482872 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43037
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:21.568992 11374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:21.569200 11374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:21.569224 11374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:21.570644 11374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:21.570698 11374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:21.570726 11374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:36:21.572278 11374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:21.572546 11374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:21.572803 11374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:21.578061 11380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:21.578074 11383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:21.578083 11381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:21.578357 11374 server_base.cc:1034] running on GCE node
I20260807 22:36:21.578596 11374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:21.578852 11374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:21.580011 11374 hybrid_clock.cc:648] HybridClock initialized: now 1786142181579983 us; error 44 us; skew 500 ppm
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:21.588282 11374 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:36:21.588789 11374 webserver.cc:460] Webserver started at http://127.28.82.193:39809/ using document root <none> and password file <none>
I20260807 22:36:21.589056 11374 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:21.589133 11374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:21.589284 11374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:21.590337 11374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "10cb53487344447bae1b5f42a8028173"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "e1fc979a1ecf220892692b62faee114a"
server_key_iv: "7764ec104b622d25dbecc5281facb95a"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.590813 11374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "10cb53487344447bae1b5f42a8028173"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "e1fc979a1ecf220892692b62faee114a"
server_key_iv: "7764ec104b622d25dbecc5281facb95a"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.592252 11374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:21.593175 11390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.593452 11374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:21.593582 11374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "10cb53487344447bae1b5f42a8028173"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "e1fc979a1ecf220892692b62faee114a"
server_key_iv: "7764ec104b622d25dbecc5281facb95a"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.593681 11374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:21.611975 11374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:21.614332 11374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:21.614547 11374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:21.614822 11374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:21.615182 11374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:21.615249 11374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.615295 11374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:21.615326 11374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.621994 11374 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:45841
I20260807 22:36:21.622051 11553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:45841 every 8 connection(s)
I20260807 22:36:21.622403 11374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:36:21.629045 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 11374
I20260807 22:36:21.629156 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:36:21.629361 29003 external_mini_cluster.cc:1428] Setting key cbd6bdb034e50822b8430148d0c43b60
I20260807 22:36:21.632977 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.623350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:44893 (local address 127.28.82.254:43037)
0807 22:36:21.623441 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:21.623443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:21.623583 (+ 140us) server_negotiation.cc:407] Connection header received
0807 22:36:21.624106 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:21.624107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:21.624138 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:21.624204 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:21.625163 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.625711 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.626820 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.627030 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.627099 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.629486 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:21.629488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:21.629488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:21.629500 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:21.631425 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.631854 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.631855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.631856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.631929 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.632243 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.632244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.632245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.632415 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:21.632489 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:21.632829 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 22:36:21.632906 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 22:36:21.633261 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.623252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43037 (local address 127.28.82.193:44893)
0807 22:36:21.623437 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:21.623487 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:21.623979 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:21.624256 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:21.624259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:21.624849 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:21.625051 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.625056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.625828 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.625830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.626661 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.626663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.627142 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.627144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.627297 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.627804 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:21.627813 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:21.629380 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:21.631518 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:21.631525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:21.631527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:21.631767 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:21.632002 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:21.632003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:21.632003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:21.632156 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:21.632497 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:21.632505 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:21.632675 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 22:36:21.632796 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 22:36:21.632974 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260807 22:36:21.633873 11556 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43037
I20260807 22:36:21.633989 11556 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:21.634179 11556 heartbeater.cc:511] Master 127.28.82.254:43037 requested a full tablet report, sending...
I20260807 22:36:21.634831 11280 ts_manager.cc:194] Registered new tserver with Master: 10cb53487344447bae1b5f42a8028173 (127.28.82.193:45841)
I20260807 22:36:21.635878 11280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:44893
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:36:21.663800 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43037
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:21.752173 11563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:21.752436 11563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:21.752480 11563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:21.753674 11563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:21.753762 11563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:21.753808 11563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:36:21.755311 11563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:21.755601 11563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:21.755863 11563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:21.761387 11572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:21.761384 11571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:21.761379 11578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:21.761477 11563 server_base.cc:1034] running on GCE node
I20260807 22:36:21.761864 11563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:21.762164 11563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:21.763351 11563 hybrid_clock.cc:648] HybridClock initialized: now 1786142181763339 us; error 50 us; skew 500 ppm
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:21.766177 11563 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:36:21.766752 11563 webserver.cc:460] Webserver started at http://127.28.82.194:39233/ using document root <none> and password file <none>
I20260807 22:36:21.767001 11563 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:21.767112 11563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:21.767257 11563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:21.768334 11563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "caef6d99411e4411bd93152280140958"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "b06299bc170f491c7e5dea86bdd4240d"
server_key_iv: "277b46dbfa2bf73b072fdcfe66aff401"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.768733 11563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "caef6d99411e4411bd93152280140958"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "b06299bc170f491c7e5dea86bdd4240d"
server_key_iv: "277b46dbfa2bf73b072fdcfe66aff401"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.770179 11563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:36:21.771219 11590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.771467 11563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:36:21.771641 11563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "caef6d99411e4411bd93152280140958"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "b06299bc170f491c7e5dea86bdd4240d"
server_key_iv: "277b46dbfa2bf73b072fdcfe66aff401"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.771729 11563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:21.788062 11563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:21.790323 11563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:21.790587 11563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:21.790886 11563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:21.791247 11563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:21.791316 11563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.791363 11563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:21.791394 11563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.798584 11563 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:40829
I20260807 22:36:21.798700 11740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:40829 every 8 connection(s)
I20260807 22:36:21.799017 11563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:36:21.801000 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 11563
I20260807 22:36:21.801131 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:36:21.801337 29003 external_mini_cluster.cc:1428] Setting key 9a48b3963d2563365477c0ac97fe0e27
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:36:21.811048 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.799928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:38827 (local address 127.28.82.254:43037)
0807 22:36:21.800014 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:21.800017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:21.800242 (+ 225us) server_negotiation.cc:407] Connection header received
0807 22:36:21.800828 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:21.800830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:21.800844 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:21.800926 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:21.802190 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.802921 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.803934 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.804244 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.804318 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.807511 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:21.807513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:21.807513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:21.807524 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:21.809357 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.809800 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.809802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.809803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.809877 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.810151 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.810152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.810153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.810312 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:21.810390 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:21.810820 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 22:36:21.810929 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:36:21.811082 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.799878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43037 (local address 127.28.82.194:38827)
0807 22:36:21.800099 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:21.800146 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:21.800637 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:21.801028 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:21.801031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:21.801867 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:21.802064 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.802069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.803047 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.803048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.803836 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.803839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.804363 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.804364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.804533 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.805212 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:21.805225 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:21.807397 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:21.809459 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:21.809466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:21.809468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:21.809709 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:21.809958 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:21.809959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:21.809959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:21.810071 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:21.810415 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:21.810421 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:21.810664 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 22:36:21.810766 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 22:36:21.810906 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":118,"threads_started":1}
I20260807 22:36:21.811722 11741 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43037
I20260807 22:36:21.811821 11741 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:21.812017 11741 heartbeater.cc:511] Master 127.28.82.254:43037 requested a full tablet report, sending...
I20260807 22:36:21.812520 11280 ts_manager.cc:194] Registered new tserver with Master: caef6d99411e4411bd93152280140958 (127.28.82.194:40829)
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
I20260807 22:36:21.813113 11280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:38827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:36:21.834154 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43037
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:36:21.918064 11753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:36:21.918269 11753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:36:21.918293 11753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:36:21.919489 11753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:36:21.919535 11753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:36:21.919562 11753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:36:21.921037 11753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:36:21.921291 11753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:36:21.921557 11753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:36:21.927052 11760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:36:21.927193 11764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:21.927330 11753 server_base.cc:1034] running on GCE node
W20260807 22:36:21.927119 11761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:36:21.927563 11753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:36:21.927788 11753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:36:21.928941 11753 hybrid_clock.cc:648] HybridClock initialized: now 1786142181928929 us; error 28 us; skew 500 ppm
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:36:21.931710 11753 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:36:21.932175 11753 webserver.cc:460] Webserver started at http://127.28.82.195:41725/ using document root <none> and password file <none>
I20260807 22:36:21.932438 11753 fs_manager.cc:359] Metadata directory not provided
I20260807 22:36:21.932487 11753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:36:21.932672 11753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:36:21.933717 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "d3a657aa5bfd4146b52ea7227b90e202"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "7934fc5341d3c99065d659bad69eea99"
server_key_iv: "ba218cdede1d75a9faf6ffd15744958a"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.934087 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "d3a657aa5bfd4146b52ea7227b90e202"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "7934fc5341d3c99065d659bad69eea99"
server_key_iv: "ba218cdede1d75a9faf6ffd15744958a"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.935564 11753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:36:21.936370 11778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.936585 11753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:36:21.936686 11753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "d3a657aa5bfd4146b52ea7227b90e202"
format_stamp: "Formatted at 2026-08-07 22:36:21 on dist-test-slave-7kzw"
server_key: "7934fc5341d3c99065d659bad69eea99"
server_key_iv: "ba218cdede1d75a9faf6ffd15744958a"
server_key_version: "encryptionkey@0"
I20260807 22:36:21.936761 11753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:36:21.944675 11753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:36:21.946614 11753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:36:21.946774 11753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:36:21.947003 11753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:36:21.947329 11753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:36:21.947367 11753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.947389 11753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:36:21.947402 11753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:36:21.953812 11753 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:37063
I20260807 22:36:21.953857 11935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:37063 every 8 connection(s)
I20260807 22:36:21.954169 11753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:36:21 dist-test-slave-7kzw krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142181, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:36:21.960678 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 11753
I20260807 22:36:21.960762 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:36:21.960933 29003 external_mini_cluster.cc:1428] Setting key 531ed6796bf9e3ba4ffc7390fcb4c0b3
I20260807 22:36:21.964010 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.954930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:52361 (local address 127.28.82.254:43037)
0807 22:36:21.955020 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:21.955022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:21.955172 (+ 150us) server_negotiation.cc:407] Connection header received
0807 22:36:21.955636 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:21.955637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:21.955647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:21.955698 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:21.956631 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.957127 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.958209 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.958406 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.958539 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.960916 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:21.960918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:21.960919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:21.960927 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:21.962698 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.963108 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.963109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.963109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.963173 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.963448 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.963449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.963450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.963586 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:21.963654 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:21.963880 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:36:21.963944 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:36:21.964231 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.954871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43037 (local address 127.28.82.195:52361)
0807 22:36:21.955047 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:21.955085 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:21.955551 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:21.955750 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:21.955753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:21.956335 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:36:21.956530 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.956535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.957242 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.957243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.958105 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.958108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.958638 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.958639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.958752 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.959250 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:36:21.959259 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:36:21.960810 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:36:21.962770 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:21.962776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:21.962778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:21.962999 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:21.963252 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:36:21.963253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:36:21.963253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:36:21.963355 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:36:21.963656 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:36:21.963661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:36:21.963780 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 22:36:21.963864 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 22:36:21.963982 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260807 22:36:21.964764 11940 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43037
I20260807 22:36:21.964880 11940 heartbeater.cc:464] Registering TS with master...
I20260807 22:36:21.965076 11940 heartbeater.cc:511] Master 127.28.82.254:43037 requested a full tablet report, sending...
I20260807 22:36:21.965479 11280 ts_manager.cc:194] Registered new tserver with Master: d3a657aa5bfd4146b52ea7227b90e202 (127.28.82.195:37063)
I20260807 22:36:21.965864 11280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:52361
I20260807 22:36:21.972980 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:36:21.981076 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.974294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33186 (local address 127.28.82.254:43037)
0807 22:36:21.974364 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:21.974367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:21.974638 (+ 271us) server_negotiation.cc:407] Connection header received
0807 22:36:21.974661 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:21.974661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:21.974671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:21.974723 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:36:21.975312 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.975863 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.976650 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.976852 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.976915 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.978120 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:36:21.978121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:36:21.978121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:36:21.978127 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:36:21.979670 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.980050 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.980051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.980052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.980107 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:36:21.980368 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:36:21.980369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:36:21.980369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:36:21.980516 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:36:21.980607 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:21.980869 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 22:36:21.980954 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:36:21.983107 11280 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:36:21.985116 11280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:36:21.991856 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.987633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:45841 (local address 127.0.0.1:56740)
0807 22:36:21.987948 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:21.987959 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:21.988017 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:21.988460 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:21.988463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:21.988465 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:36:21.988535 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.988537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.989518 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.989520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.990695 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.990698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.991537 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.991538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.991630 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.991645 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 22:36:21.991719 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:36:21.991768 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260807 22:36:21.991904 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.987771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56740 (local address 127.28.82.193:45841)
0807 22:36:21.988183 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:21.988188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:21.988199 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:36:21.988241 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:21.988243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:21.988259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:21.988341 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:36:21.988620 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.989414 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.990784 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.991437 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.991517 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.991575 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:21.991758 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 22:36:21.991806 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":83,"threads_started":1}
I20260807 22:36:21.992103 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.987634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:37063 (local address 127.0.0.1:34588)
0807 22:36:21.987797 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:21.987836 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:21.987898 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:21.988497 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:21.988498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:21.988500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:36:21.988553 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.988555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.989974 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.989975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.990980 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.990982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.991878 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.991880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.991949 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.991954 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:36:21.991991 (+ 37us) client_negotiation.cc:240] Negotiation successful
0807 22:36:21.992033 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":59,"threads_started":1}
I20260807 22:36:21.992401 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.987745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.28.82.195:37063)
0807 22:36:21.988150 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:21.988155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:21.988165 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:36:21.988228 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:21.988231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:21.988248 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:21.988631 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:36:21.988721 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.989855 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.991065 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.991776 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.992160 (+ 384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.992207 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:21.992260 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 22:36:21.992305 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":90,"threads_started":1}
I20260807 22:36:21.992935 11453 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4 (DEFAULT_TABLE table=test-table [id=c3f814f722f24baa869a0c2d729f1cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:36:21.993036 11851 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4 (DEFAULT_TABLE table=test-table [id=c3f814f722f24baa869a0c2d729f1cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:36:21.993225 11960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.987868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:40829 (local address 127.0.0.1:41138)
0807 22:36:21.988025 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:21.988034 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:21.988076 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:21.988576 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:21.988577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:21.988579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:36:21.988621 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.988623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.991050 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.991051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.991870 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:21.991872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.993007 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.993009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:21.993066 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.993070 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:36:21.993122 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 22:36:21.993161 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260807 22:36:21.993372 11453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:36:21.993381 11851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:36:21.993517 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:21.988111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.28.82.194:40829)
0807 22:36:21.988347 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:21.988352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:21.988363 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:36:21.988405 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:21.988407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:21.988422 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:21.988496 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:36:21.988710 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.989851 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.991971 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:21.992845 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:21.992917 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:21.992968 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:21.993384 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 22:36:21.993432 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":109,"threads_started":1}
I20260807 22:36:21.995129 11654 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4 (DEFAULT_TABLE table=test-table [id=c3f814f722f24baa869a0c2d729f1cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:36:21.995476 11654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:36:21.996220 11962 tablet_bootstrap.cc:492] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173: Bootstrap starting.
I20260807 22:36:21.997022 11962 tablet_bootstrap.cc:654] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173: Neither blocks nor log segments found. Creating new log.
I20260807 22:36:21.997622 11962 log.cc:824] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173: Log is configured to *not* fsync() on all Append() calls
I20260807 22:36:21.998399 11962 tablet_bootstrap.cc:492] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173: No bootstrap required, opened a new log
I20260807 22:36:21.998607 11966 tablet_bootstrap.cc:492] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Bootstrap starting.
I20260807 22:36:21.998526 11962 ts_tablet_manager.cc:1397] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:36:21.998694 11963 tablet_bootstrap.cc:492] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: Bootstrap starting.
I20260807 22:36:21.999215 11963 tablet_bootstrap.cc:654] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: Neither blocks nor log segments found. Creating new log.
I20260807 22:36:21.999270 11966 tablet_bootstrap.cc:654] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Neither blocks nor log segments found. Creating new log.
I20260807 22:36:21.999847 11963 log.cc:824] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: Log is configured to *not* fsync() on all Append() calls
I20260807 22:36:21.999878 11966 log.cc:824] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Log is configured to *not* fsync() on all Append() calls
I20260807 22:36:22.000554 11963 tablet_bootstrap.cc:492] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: No bootstrap required, opened a new log
I20260807 22:36:22.000643 11963 ts_tablet_manager.cc:1397] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:36:22.000634 11962 raft_consensus.cc:348] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.000854 11962 raft_consensus.cc:374] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:36:22.000881 11962 raft_consensus.cc:729] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10cb53487344447bae1b5f42a8028173, State: Initialized, Role: FOLLOWER
I20260807 22:36:22.001024 11962 consensus_queue.cc:260] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.001317 11962 ts_tablet_manager.cc:1428] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 22:36:22.001396 11556 heartbeater.cc:503] Master 127.28.82.254:43037 was elected leader, sending a full tablet report...
I20260807 22:36:22.001904 11966 tablet_bootstrap.cc:492] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: No bootstrap required, opened a new log
I20260807 22:36:22.001955 11966 ts_tablet_manager.cc:1397] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 22:36:22.002756 11963 raft_consensus.cc:348] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.002918 11963 raft_consensus.cc:374] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:36:22.002962 11963 raft_consensus.cc:729] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3a657aa5bfd4146b52ea7227b90e202, State: Initialized, Role: FOLLOWER
I20260807 22:36:22.003116 11963 consensus_queue.cc:260] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.003453 11963 ts_tablet_manager.cc:1428] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:36:22.003532 11940 heartbeater.cc:503] Master 127.28.82.254:43037 was elected leader, sending a full tablet report...
I20260807 22:36:22.003611 11966 raft_consensus.cc:348] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.003777 11966 raft_consensus.cc:374] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:36:22.003855 11966 raft_consensus.cc:729] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caef6d99411e4411bd93152280140958, State: Initialized, Role: FOLLOWER
I20260807 22:36:22.003989 11966 consensus_queue.cc:260] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.004287 11966 ts_tablet_manager.cc:1428] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:36:22.004328 11741 heartbeater.cc:503] Master 127.28.82.254:43037 was elected leader, sending a full tablet report...
W20260807 22:36:22.049791 11747 tablet.cc:2369] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:36:22.076913 11972 raft_consensus.cc:482] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:36:22.077051 11972 raft_consensus.cc:504] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.077802 11972 leader_election.cc:288] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3a657aa5bfd4146b52ea7227b90e202 (127.28.82.195:37063), 10cb53487344447bae1b5f42a8028173 (127.28.82.193:45841)
I20260807 22:36:22.080785 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:22.078115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:54177 (local address 127.28.82.195:37063)
0807 22:36:22.078188 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:22.078191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:22.078203 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:36:22.078234 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:22.078235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:22.078246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:22.078346 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:36:22.078576 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.079066 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.080013 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.080394 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.080496 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:22.080505 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:22.080667 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:36:22.080702 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:36:22.080821 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:22.077991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:37063 (local address 127.28.82.194:54177)
0807 22:36:22.078041 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:22.078056 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:22.078108 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:22.078352 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:22.078354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:22.078356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:36:22.078432 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:22.078433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.079162 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.079163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:22.079907 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:22.079909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.080493 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.080494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:22.080582 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:22.080585 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:36:22.080660 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 22:36:22.080701 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 22:36:22.080817 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:22.078204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:44523 (local address 127.28.82.193:45841)
0807 22:36:22.078294 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:22.078296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:22.078307 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:36:22.078380 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:22.078381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:22.078392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:22.078461 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:36:22.078656 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.079139 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.080068 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.080440 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.080535 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:22.080544 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:22.080708 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 22:36:22.080746 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 22:36:22.081027 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:22.078028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:45841 (local address 127.28.82.194:44523)
0807 22:36:22.078236 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:22.078260 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 22:36:22.078311 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:36:22.078499 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:36:22.078500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:36:22.078503 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:36:22.078579 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:22.078581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.079237 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.079238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:22.079968 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:36:22.079971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.080534 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.080535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:36:22.080608 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:22.080612 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:36:22.080694 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:36:22.080734 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":126,"threads_started":1}
I20260807 22:36:22.081220 11872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6516dcc85d4f1eb5ba7b19d53406f4" candidate_uuid: "caef6d99411e4411bd93152280140958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3a657aa5bfd4146b52ea7227b90e202" is_pre_election: true
I20260807 22:36:22.081393 11872 raft_consensus.cc:2454] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caef6d99411e4411bd93152280140958 in term 0.
I20260807 22:36:22.081410 11483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6516dcc85d4f1eb5ba7b19d53406f4" candidate_uuid: "caef6d99411e4411bd93152280140958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10cb53487344447bae1b5f42a8028173" is_pre_election: true
I20260807 22:36:22.081602 11483 raft_consensus.cc:2454] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caef6d99411e4411bd93152280140958 in term 0.
I20260807 22:36:22.081686 11594 leader_election.cc:302] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caef6d99411e4411bd93152280140958, d3a657aa5bfd4146b52ea7227b90e202; no voters:
I20260807 22:36:22.081861 11972 raft_consensus.cc:2781] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:36:22.081931 11972 raft_consensus.cc:482] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:36:22.081964 11972 raft_consensus.cc:3037] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:36:22.082792 11972 raft_consensus.cc:504] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.082952 11972 leader_election.cc:288] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [CANDIDATE]: Term 1 election: Requested vote from peers d3a657aa5bfd4146b52ea7227b90e202 (127.28.82.195:37063), 10cb53487344447bae1b5f42a8028173 (127.28.82.193:45841)
I20260807 22:36:22.083150 11872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6516dcc85d4f1eb5ba7b19d53406f4" candidate_uuid: "caef6d99411e4411bd93152280140958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3a657aa5bfd4146b52ea7227b90e202"
I20260807 22:36:22.083148 11483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6516dcc85d4f1eb5ba7b19d53406f4" candidate_uuid: "caef6d99411e4411bd93152280140958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10cb53487344447bae1b5f42a8028173"
I20260807 22:36:22.083220 11872 raft_consensus.cc:3037] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:36:22.083252 11483 raft_consensus.cc:3037] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:36:22.083935 11872 raft_consensus.cc:2454] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caef6d99411e4411bd93152280140958 in term 1.
I20260807 22:36:22.084170 11483 raft_consensus.cc:2454] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caef6d99411e4411bd93152280140958 in term 1.
I20260807 22:36:22.084201 11594 leader_election.cc:302] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caef6d99411e4411bd93152280140958, d3a657aa5bfd4146b52ea7227b90e202; no voters:
I20260807 22:36:22.084327 11972 raft_consensus.cc:2781] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:36:22.084542 11972 raft_consensus.cc:686] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 1 LEADER]: Becoming Leader. State: Replica: caef6d99411e4411bd93152280140958, State: Running, Role: LEADER
I20260807 22:36:22.084693 11972 consensus_queue.cc:237] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } }
I20260807 22:36:22.085549 11279 catalog_manager.cc:5495] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 reported cstate change: term changed from 0 to 1, leader changed from <none> to caef6d99411e4411bd93152280140958 (127.28.82.194). New cstate: current_term: 1 leader_uuid: "caef6d99411e4411bd93152280140958" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caef6d99411e4411bd93152280140958" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40829 } health_report { overall_health: HEALTHY } } }
I20260807 22:36:22.112711 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:22.110198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.28.82.194:40829)
0807 22:36:22.110275 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 22:36:22.110278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:36:22.110285 (+ 7us) server_negotiation.cc:407] Connection header received
0807 22:36:22.110317 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:36:22.110318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:36:22.110336 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:36:22.110423 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:36:22.110565 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.111043 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.111867 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:36:22.112084 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:36:22.112169 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:36:22.112339 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:36:22.112438 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:36:22.112541 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:36:22.112567 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 22:36:22.112595 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:36:22.114912 11483 raft_consensus.cc:1261] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 1 FOLLOWER]: Refusing update from remote peer caef6d99411e4411bd93152280140958: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:36:22.114955 11872 raft_consensus.cc:1261] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 1 FOLLOWER]: Refusing update from remote peer caef6d99411e4411bd93152280140958: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:36:22.115350 11972 consensus_queue.cc:1035] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3a657aa5bfd4146b52ea7227b90e202" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 37063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:36:22.115543 11975 consensus_queue.cc:1035] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10cb53487344447bae1b5f42a8028173" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:36:22.118193 11981 mvcc.cc:205] Tried to move back new op lower bound from 7316038377939292160 to 7316038377819488256. Current Snapshot: MvccSnapshot[applied={T|T < 7316038377939292160}]
I20260807 22:36:22.120527 11279 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:36:22.120633 11279 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:36:22.121587 11279 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 701a06dbcfff431f997d79e43641cd91: Sending DeleteTablet for 3 replicas of tablet 4a6516dcc85d4f1eb5ba7b19d53406f4
I20260807 22:36:22.121968 11851 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:36:22 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:34588
I20260807 22:36:22.121992 11453 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:36:22 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:56740
I20260807 22:36:22.122046 11654 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:36:22 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:41138
I20260807 22:36:22.122264 11994 tablet_replica.cc:330] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: stopping tablet replica
I20260807 22:36:22.122275 11991 tablet_replica.cc:330] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173: stopping tablet replica
I20260807 22:36:22.122339 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 11374
I20260807 22:36:22.122421 11991 raft_consensus.cc:2229] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:36:22.122586 11991 raft_consensus.cc:2258] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P 10cb53487344447bae1b5f42a8028173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:36:22.122394 11994 raft_consensus.cc:2229] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:36:22.122957 11994 raft_consensus.cc:2258] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:36:22.123098 11990 tablet_replica.cc:330] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: stopping tablet replica
I20260807 22:36:22.123267 11990 raft_consensus.cc:2229] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:36:22.123777 11994 ts_tablet_manager.cc:1905] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:36:22.125469 11994 ts_tablet_manager.cc:1918] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:36:22.125563 11994 log.cc:1196] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/4a6516dcc85d4f1eb5ba7b19d53406f4
I20260807 22:36:22.125893 11994 ts_tablet_manager.cc:1939] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P caef6d99411e4411bd93152280140958: Deleting consensus metadata
I20260807 22:36:22.126332 11263 catalog_manager.cc:4841] TS caef6d99411e4411bd93152280140958 (127.28.82.194:40829): tablet 4a6516dcc85d4f1eb5ba7b19d53406f4 (table test-table [id=c3f814f722f24baa869a0c2d729f1cf4]) successfully deleted
I20260807 22:36:22.127108 11990 raft_consensus.cc:2258] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:36:22.127566 11990 ts_tablet_manager.cc:1905] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:36:22.129311 11990 ts_tablet_manager.cc:1918] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:36:22.129391 11990 log.cc:1196] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/4a6516dcc85d4f1eb5ba7b19d53406f4
I20260807 22:36:22.129694 11990 ts_tablet_manager.cc:1939] T 4a6516dcc85d4f1eb5ba7b19d53406f4 P d3a657aa5bfd4146b52ea7227b90e202: Deleting consensus metadata
I20260807 22:36:22.130216 11262 catalog_manager.cc:4841] TS d3a657aa5bfd4146b52ea7227b90e202 (127.28.82.195:37063): tablet 4a6516dcc85d4f1eb5ba7b19d53406f4 (table test-table [id=c3f814f722f24baa869a0c2d729f1cf4]) successfully deleted
W20260807 22:36:22.136221 11263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:45841 (error 108)
I20260807 22:36:22.136895 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 11563
I20260807 22:36:22.136976 11960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:36:22.136762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:45841 (local address 127.0.0.1:56754)
0807 22:36:22.136868 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 22:36:22.136916 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:45841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 22:36:22.139649 11263 catalog_manager.cc:4584] TS 10cb53487344447bae1b5f42a8028173 (127.28.82.193:45841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a6516dcc85d4f1eb5ba7b19d53406f4: Network error: Client connection negotiation failed: client connection to 127.28.82.193:45841: connect: Connection refused (error 111)
I20260807 22:36:22.147794 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 11753
I20260807 22:36:22.156775 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 11238
2026-08-07T22:36:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2239 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:36:22 dist-test-slave-7kzw krb5kdc[11999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:36:22 dist-test-slave-7kzw krb5kdc[11999](info): set up 2 sockets
Aug 07 22:36:22 dist-test-slave-7kzw krb5kdc[11999](info): commencing operation
krb5kdc: starting...
W20260807 22:36:23.396341 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:36:23 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:36:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:36:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.82.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.82.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/rangeradmin_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/rangeradmin_127.28.82.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.82.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.82.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/rangerlookup_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/rangerlookup_127.28.82.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.82.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.82.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/HTTP_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/HTTP_127.28.82.212@KRBTEST.COM.keytab.
I20260807 22:36:23.585938 29003 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskj8JEAI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/postgres -l logfile start
2026-08-07 22:36:26.539 UTC [12040] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 22:36:26.539 UTC [12040] LOG: listening on IPv4 address "127.28.82.212", port 53541
2026-08-07 22:36:26.543 UTC [12040] LOG: listening on Unix socket "/tmp/.s.PGSQL.53541"
2026-08-07 22:36:26.567 UTC [12044] LOG: database system was shut down at 2026-08-07 22:36:24 UTC
2026-08-07 22:36:26.572 UTC [12040] LOG: database system is ready to accept connections
I20260807 22:36:27.702625 29003 mini_postgres.cc:96] Postgres bound to 53541
2026-08-07 22:36:27.712 UTC [12055] FATAL: database "slave" does not exist
127.28.82.212:53541 - accepting connections
I20260807 22:36:27.713092 29003 mini_ranger.cc:161] Starting Ranger...
I20260807 22:36:27.731261 29003 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 22:36:28.330825 29003 mini_ranger.cc:87] Created ranger Postgres database
I20260807 22:36:28.330919 29003 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-admin
2026-08-07 22:36:28,668 [I] DB FLAVOR :POSTGRES
2026-08-07 22:36:28,668 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:36:28,668 [I] Checking connection..
2026-08-07 22:36:28,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 22:36:28,936 [I] Checking connection passed.
2026-08-07 22:36:28,936 [I] --------- Verifying version history table ---------
2026-08-07 22:36:28,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:36:29,201 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:36:29,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:36:29,474 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:36:29,474 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 22:36:29,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 22:36:29.733 UTC [12145] WARNING: there is no transaction in progress
2026-08-07 22:36:29,742 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 22:36:29,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:36:30,016 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 22:36:30,017 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:36:30,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 22:36:30,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 22:36:30,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7kzw.c.gcp-upstream.internal','N') ;"
2026-08-07 22:36:30,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:36:31,071 [I] Table x_portal_user does not exist in database ranger
2026-08-07 22:36:31,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:36:31,347 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 22:36:31,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:36:31,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:36:31,872 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 22:36:31,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 22:36:32.918 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:32.937 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:32.956 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:32.974 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:32.992 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.011 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.075 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.082 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.097 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.108 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.120 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.134 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.145 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.152 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.159 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33.167 UTC [12296] WARNING: there is no transaction in progress
2026-08-07 22:36:33,901 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 22:36:33,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 22:36:34,173 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 22:36:34,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:36:34,430 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 22:36:34,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:36:34,725 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 22:36:34,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:36:35,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:36:35,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7kzw.c.gcp-upstream.internal';"
2026-08-07 22:36:35,581 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 22:36:35,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 22:36:35,854 [I] DB_PATCHES have already been applied
I20260807 22:36:35.860682 29003 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-admin:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskj8JEAI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 22:36:35.860749 29003 mini_ranger.cc:193] Using host: 127.28.82.212
I20260807 22:36:35.864874 29003 mini_ranger.cc:227] Ranger admin URL: http://127.28.82.212:46947
Aug 07, 2026 10:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 22:36:37.302325 29003 mini_ranger.cc:160] Time spent starting Ranger: real 9.589s user 0.000s sys 0.012s
Aug 07, 2026 10:36:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:36:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 10:36:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:36:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:36:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 22:36:58.598526 29003 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.82.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.82.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/rangerkms_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/rangerkms_127.28.82.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.82.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.82.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.82.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/HTTP_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.82.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/HTTP_127.28.82.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 22:36:58 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142218, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 22:36:58.692878 29003 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 22:36:58.707597 29003 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 22:36:59.599267 29003 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 22:36:59.599350 29003 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms
I20260807 22:36:59.599364 29003 mini_ranger_kms.cc:227] Using postgres at 127.28.82.212:53541
2026-08-07 22:37:00,278 [I] DB FLAVOR :POSTGRES
2026-08-07 22:37:00,279 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:37:00,279 [I] Checking connection
2026-08-07 22:37:00,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 22:37:00,662 [I] connection success
2026-08-07 22:37:00,662 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 22:37:00,663 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 22:37:00,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 22:37:00,928 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 22:37:00,928 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:37:00,929 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 22:37:00,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:53541/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 22:37:01,215 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 22:37:01.640496 29003 mini_ranger_kms.cc:310] Created kms service
I20260807 22:37:01.853488 29003 mini_ranger_kms.cc:326] Created kudu user
I20260807 22:37:01.897934 29003 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 22:37:01 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142218, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:37:02.301932 29003 mini_ranger_kms.cc:384] Added ranger policy
I20260807 22:37:02.302063 29003 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms:/tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 22:37:02.302084 29003 mini_ranger_kms.cc:240] Using host: 127.28.82.212
I20260807 22:37:02.306572 29003 mini_ranger_kms.cc:276] Ranger KMS PID: 12537
I20260807 22:37:02.306713 29003 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.82.212:48455
Aug 07, 2026 10:37:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:37:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:37:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:37:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:37:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:37:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 22:37:05.189134 29003 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.496s user 0.000s sys 0.642s
I20260807 22:37:05.189261 29003 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 22:37:05.189347 29003 mini_ranger_kms.cc:401] 127.28.82.212:48455/kms/v1/keys
Aug 07 22:37:05 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142225, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.82.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 22:37:05 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142225, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.82.212@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
Aug 07 22:37:06 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142225, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.82.212@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
Aug 07 22:37:09 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:37:09.138720 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43823
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:43823
--ranger_config_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:48455/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:37:09.264060 12621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:37:09.264292 12621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:37:09.264317 12621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:37:09.265578 12621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:37:09.265630 12621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:37:09.265646 12621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:37:09.265659 12621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:37:09.265671 12621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:37:09.267738 12621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.82.212:48455/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:43823
--ranger_config_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:37:09.268034 12621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:37:09.268301 12621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:37:09.274977 12626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:37:09.275113 12621 server_base.cc:1034] running on GCE node
W20260807 22:37:09.275058 12627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:37:09.275121 12632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:37:09.275524 12621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:37:09.275940 12621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:37:09.277132 12621 hybrid_clock.cc:648] HybridClock initialized: now 1786142229277108 us; error 43 us; skew 500 ppm
Aug 07 22:37:09 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142229, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:37:09.280596 12621 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:37:09.281047 12621 webserver.cc:460] Webserver started at http://127.28.82.254:44229/ using document root <none> and password file <none>
I20260807 22:37:09.281397 12621 fs_manager.cc:359] Metadata directory not provided
I20260807 22:37:09.281489 12621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:37:09.281700 12621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:37:09 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142229, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:37:09.336897 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "8e2adfa8a35d42ac8dbf7c47bc840638"
format_stamp: "Formatted at 2026-08-07 22:37:09 on dist-test-slave-7kzw"
server_key: "92d321d5dbf6fca15701d8425b79e112"
server_key_iv: "bab703307ab76de1dce069ea30ea9bc1"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:09.337455 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "8e2adfa8a35d42ac8dbf7c47bc840638"
format_stamp: "Formatted at 2026-08-07 22:37:09 on dist-test-slave-7kzw"
server_key: "92d321d5dbf6fca15701d8425b79e112"
server_key_iv: "bab703307ab76de1dce069ea30ea9bc1"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:09.339556 12621 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 22:37:09.352476 12642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:09.352728 12621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:37:09.352856 12621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "8e2adfa8a35d42ac8dbf7c47bc840638"
format_stamp: "Formatted at 2026-08-07 22:37:09 on dist-test-slave-7kzw"
server_key: "92d321d5dbf6fca15701d8425b79e112"
server_key_iv: "bab703307ab76de1dce069ea30ea9bc1"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:09.352981 12621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:37:09.372215 12621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:37:09.374300 12621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:37:09.374600 12621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:37:09.386747 12621 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:43823
I20260807 22:37:09.386878 12706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:43823 every 8 connection(s)
I20260807 22:37:09.387339 12621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:37:09.387759 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 12621
I20260807 22:37:09.387871 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:37:09.388070 29003 external_mini_cluster.cc:1428] Setting key b8f90bfff1dcd68b7d2bf2687153cb38
I20260807 22:37:09.388381 12707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:37:09.393904 12707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638: Bootstrap starting.
I20260807 22:37:09.395352 12707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638: Neither blocks nor log segments found. Creating new log.
I20260807 22:37:09.396298 12707 log.cc:824] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638: Log is configured to *not* fsync() on all Append() calls
I20260807 22:37:09.397172 12707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638: No bootstrap required, opened a new log
Aug 07 22:37:09 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:37:09.400906 12707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2adfa8a35d42ac8dbf7c47bc840638" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43823 } }
I20260807 22:37:09.401074 12707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:37:09.401157 12707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2adfa8a35d42ac8dbf7c47bc840638, State: Initialized, Role: FOLLOWER
I20260807 22:37:09.401335 12707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2adfa8a35d42ac8dbf7c47bc840638" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43823 } }
I20260807 22:37:09.401440 12707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:37:09.401474 12707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:37:09.401558 12707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:37:09.402565 12707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2adfa8a35d42ac8dbf7c47bc840638" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43823 } }
I20260807 22:37:09.402685 12707 leader_election.cc:302] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e2adfa8a35d42ac8dbf7c47bc840638; no voters:
I20260807 22:37:09.402881 12707 leader_election.cc:288] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:37:09.403041 12715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:37:09.403258 12707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:37:09.403270 12715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [term 1 LEADER]: Becoming Leader. State: Replica: 8e2adfa8a35d42ac8dbf7c47bc840638, State: Running, Role: LEADER
I20260807 22:37:09.403420 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:09.388677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.28.82.254:43823)
0807 22:37:09.388986 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:09.388995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:09.389064 (+ 69us) server_negotiation.cc:407] Connection header received
0807 22:37:09.390369 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:09.390380 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:09.390446 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:09.390682 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:37:09.391456 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:09.393348 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:09.394529 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:09.395485 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:09.395759 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:09.399057 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:37:09.399062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:37:09.399063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:37:09.399075 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:37:09.401231 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:09.401696 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:09.401699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:09.401699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:09.401822 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:09.402072 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:09.402073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:09.402074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:09.402403 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:37:09.402585 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:09.402975 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 22:37:09.403109 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":88,"threads_started":1}
I20260807 22:37:09.403460 12715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2adfa8a35d42ac8dbf7c47bc840638" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43823 } }
I20260807 22:37:09.404048 12715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e2adfa8a35d42ac8dbf7c47bc840638. Latest consensus state: current_term: 1 leader_uuid: "8e2adfa8a35d42ac8dbf7c47bc840638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2adfa8a35d42ac8dbf7c47bc840638" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43823 } } }
I20260807 22:37:09.404122 12715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [sys.catalog]: This master's current role is: LEADER
I20260807 22:37:09.405139 12719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:37:09.405457 12717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e2adfa8a35d42ac8dbf7c47bc840638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2adfa8a35d42ac8dbf7c47bc840638" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43823 } } }
I20260807 22:37:09.405597 12717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638 [sys.catalog]: This master's current role is: LEADER
I20260807 22:37:09.406045 12719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:37:09.409166 12707 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:37:09.418596 12719 catalog_manager.cc:1347] Generated new cluster ID: 454331af168140ca9f1c7bcdc520433f
I20260807 22:37:09.418895 12719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:37:09.447072 12719 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:37:09.447965 12719 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:37:09.454962 12719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638: Generated new TSK 0
I20260807 22:37:09.455245 12719 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:37:10.473281 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:48455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43823
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:37:10.611253 12768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:37:10.611500 12768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:37:10.611544 12768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:37:10.612824 12768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:37:10.612888 12768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:37:10.612931 12768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:37:10.615804 12768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.82.212:48455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:37:10.616117 12768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:37:10.616470 12768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:37:10.625190 12775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:37:10.625299 12773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:37:10.625576 12778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:37:10.625679 12768 server_base.cc:1034] running on GCE node
I20260807 22:37:10.625842 12768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:37:10.626102 12768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:37:10.627280 12768 hybrid_clock.cc:648] HybridClock initialized: now 1786142230627260 us; error 36 us; skew 500 ppm
Aug 07 22:37:10 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142230, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:37:10.630625 12768 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:37:10.631059 12768 webserver.cc:460] Webserver started at http://127.28.82.193:43679/ using document root <none> and password file <none>
I20260807 22:37:10.631520 12768 fs_manager.cc:359] Metadata directory not provided
I20260807 22:37:10.631714 12768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:37:10.631899 12768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:37:10 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142230, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:37:10.648550 12768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "80373d338ec74634b95e00d73a3ffc75"
format_stamp: "Formatted at 2026-08-07 22:37:10 on dist-test-slave-7kzw"
server_key: "ea71995dd641c07bb2024acc1c84e199"
server_key_iv: "292af51917d5bbea1946acfc90fa550c"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:10.649024 12768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "80373d338ec74634b95e00d73a3ffc75"
format_stamp: "Formatted at 2026-08-07 22:37:10 on dist-test-slave-7kzw"
server_key: "ea71995dd641c07bb2024acc1c84e199"
server_key_iv: "292af51917d5bbea1946acfc90fa550c"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:10.652037 12768 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 22:37:10.676805 12791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:10.677069 12768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 22:37:10.677182 12768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "80373d338ec74634b95e00d73a3ffc75"
format_stamp: "Formatted at 2026-08-07 22:37:10 on dist-test-slave-7kzw"
server_key: "ea71995dd641c07bb2024acc1c84e199"
server_key_iv: "292af51917d5bbea1946acfc90fa550c"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:10.677313 12768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:37:10.691186 12768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:37:10.695267 12768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:37:10.695604 12768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:37:10.696112 12768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:37:10.696619 12768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:37:10.696748 12768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:10.696786 12768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:37:10.696815 12768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:10.719139 12768 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:42321
I20260807 22:37:10.719748 12768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:37:10.720626 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 12768
I20260807 22:37:10.720753 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:37:10.721033 29003 external_mini_cluster.cc:1428] Setting key c05bb377fc6bea51982860e636aecbb3
I20260807 22:37:10.730778 12944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:42321 every 8 connection(s)
Aug 07 22:37:10 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142230, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:37:10.736941 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:10.720866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43823 (local address 127.28.82.193:40413)
0807 22:37:10.721144 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 22:37:10.721202 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:37:10.722385 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:37:10.722814 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:37:10.722819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:37:10.723346 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:37:10.723790 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:10.723796 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:10.725239 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:10.725240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:10.726304 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:10.726307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:10.726938 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:10.726940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:10.727175 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:10.729596 (+ 2421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:37:10.729606 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:37:10.731746 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:37:10.734625 (+ 2879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:37:10.734634 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:37:10.734637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:37:10.735020 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:37:10.735324 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:37:10.735325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:37:10.735326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:37:10.735490 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:37:10.735913 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:37:10.735921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:37:10.736178 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 22:37:10.736307 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 22:37:10.736502 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260807 22:37:10.737588 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:10.721341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:40413 (local address 127.28.82.254:43823)
0807 22:37:10.722593 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:10.722595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:10.722602 (+ 7us) server_negotiation.cc:407] Connection header received
0807 22:37:10.722634 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:10.722635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:10.722648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:10.723917 (+ 1269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:37:10.723979 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:10.725040 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:10.726394 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:10.726780 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:10.727535 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:10.731898 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:37:10.731899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:37:10.731900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:37:10.731909 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:37:10.734437 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:10.735137 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:10.735138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:10.735139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:10.735220 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:10.735598 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:10.735599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:10.735600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:10.735804 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:37:10.735868 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:10.737377 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0807 22:37:10.737474 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1227,"thread_start_us":89,"threads_started":1}
I20260807 22:37:10.738140 12945 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43823
I20260807 22:37:10.738312 12945 heartbeater.cc:464] Registering TS with master...
I20260807 22:37:10.738597 12945 heartbeater.cc:511] Master 127.28.82.254:43823 requested a full tablet report, sending...
I20260807 22:37:10.739454 12665 ts_manager.cc:194] Registered new tserver with Master: 80373d338ec74634b95e00d73a3ffc75 (127.28.82.193:42321)
I20260807 22:37:10.740607 12665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:40413
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:37:10.762108 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:48455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43823
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:37:10.910766 12953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:37:10.911144 12953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:37:10.911233 12953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:37:10.913170 12953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:37:10.913296 12953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:37:10.913391 12953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:37:10.916435 12953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.82.212:48455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:37:10.916985 12953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:37:10.917357 12953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:37:10.927592 12966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:37:10.928851 12964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:37:10.929239 12953 server_base.cc:1034] running on GCE node
W20260807 22:37:10.929781 12968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:37:10.930114 12953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:37:10.930430 12953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:37:10.931664 12953 hybrid_clock.cc:648] HybridClock initialized: now 1786142230931628 us; error 39 us; skew 500 ppm
Aug 07 22:37:10 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142230, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:37:10.936671 12953 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:37:10.937286 12953 webserver.cc:460] Webserver started at http://127.28.82.194:46645/ using document root <none> and password file <none>
I20260807 22:37:10.937628 12953 fs_manager.cc:359] Metadata directory not provided
I20260807 22:37:10.937752 12953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:37:10.938004 12953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:37:10 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142230, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:37:10.958659 12953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "17656cf5bcea4822a911021ffbe58e98"
format_stamp: "Formatted at 2026-08-07 22:37:10 on dist-test-slave-7kzw"
server_key: "65106046c8fb5e80d96f243b18d216f2"
server_key_iv: "ae37a72adef2c1fdb27b371e289b4745"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:10.959290 12953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "17656cf5bcea4822a911021ffbe58e98"
format_stamp: "Formatted at 2026-08-07 22:37:10 on dist-test-slave-7kzw"
server_key: "65106046c8fb5e80d96f243b18d216f2"
server_key_iv: "ae37a72adef2c1fdb27b371e289b4745"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:10.962756 12953 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 22:37:10.994736 12977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:10.995265 12953 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.001s
I20260807 22:37:10.995451 12953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "17656cf5bcea4822a911021ffbe58e98"
format_stamp: "Formatted at 2026-08-07 22:37:10 on dist-test-slave-7kzw"
server_key: "65106046c8fb5e80d96f243b18d216f2"
server_key_iv: "ae37a72adef2c1fdb27b371e289b4745"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:10.995618 12953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:37:11.016885 12953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:37:11.019771 12953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:37:11.020004 12953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:37:11.020596 12953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:37:11.021171 12953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:37:11.021277 12953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:11.021361 12953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:37:11.021431 12953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:11.034183 12953 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:40119
I20260807 22:37:11.034754 12953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:37:11.035120 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 12953
I20260807 22:37:11.035245 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:37:11.035430 29003 external_mini_cluster.cc:1428] Setting key 4f3a4a6ce2d174aaf3450e1132f83cd8
I20260807 22:37:11.036027 13122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:40119 every 8 connection(s)
Aug 07 22:37:11 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142230, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
Aug 07 22:37:11 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142231, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:37:11.049793 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.038789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:56567 (local address 127.28.82.254:43823)
0807 22:37:11.038833 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.038835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.038842 (+ 7us) server_negotiation.cc:407] Connection header received
0807 22:37:11.038869 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.038869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.038884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.038945 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:37:11.039947 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.040769 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.042450 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.042791 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.042836 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.046120 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:37:11.046122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:37:11.046122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:37:11.046131 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:37:11.048018 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:11.048580 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:11.048581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:11.048582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:11.048644 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:11.048957 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:11.048958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:11.048958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:11.049106 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:37:11.049154 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.049565 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.049661 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:37:11.050383 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.036696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43823 (local address 127.28.82.194:56567)
0807 22:37:11.037301 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 22:37:11.037346 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:37:11.037960 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:37:11.039084 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:37:11.039089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:37:11.039472 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:37:11.039826 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.039832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.040888 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.040889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.042346 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.042348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.042932 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.042932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.043160 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.043887 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:37:11.043899 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:37:11.045997 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:37:11.048124 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:37:11.048132 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:37:11.048135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:37:11.048480 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:37:11.048718 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:37:11.048718 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:37:11.048719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:37:11.048870 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:37:11.049218 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:37:11.049225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:37:11.049395 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 22:37:11.049952 (+ 557us) client_negotiation.cc:240] Negotiation successful
0807 22:37:11.050122 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":184,"threads_started":1}
I20260807 22:37:11.050915 13123 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43823
I20260807 22:37:11.051028 13123 heartbeater.cc:464] Registering TS with master...
I20260807 22:37:11.051244 13123 heartbeater.cc:511] Master 127.28.82.254:43823 requested a full tablet report, sending...
I20260807 22:37:11.051640 12665 ts_manager.cc:194] Registered new tserver with Master: 17656cf5bcea4822a911021ffbe58e98 (127.28.82.194:40119)
I20260807 22:37:11.052110 12665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:56567
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:37:11.081949 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:48455/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43823
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:37:11.216763 13129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:37:11.217047 13129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:37:11.217103 13129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:37:11.218750 13129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:37:11.218827 13129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:37:11.218881 13129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:37:11.220837 13129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.82.212:48455/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:43823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:37:11.221156 13129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:37:11.221415 13129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:37:11.229419 13140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:37:11.229614 13138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:37:11.229277 13135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:37:11.230443 13129 server_base.cc:1034] running on GCE node
I20260807 22:37:11.230752 13129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:37:11.231046 13129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:37:11.232237 13129 hybrid_clock.cc:648] HybridClock initialized: now 1786142231232205 us; error 50 us; skew 500 ppm
Aug 07 22:37:11 dist-test-slave-7kzw krb5kdc[11999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142231, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:37:11.236595 13129 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:37:11.237066 13129 webserver.cc:460] Webserver started at http://127.28.82.195:34571/ using document root <none> and password file <none>
I20260807 22:37:11.237340 13129 fs_manager.cc:359] Metadata directory not provided
I20260807 22:37:11.237432 13129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:37:11.237576 13129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:37:11 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142231, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:37:11.261518 13129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "ad4d21b4e2cc4929889396c44e2d9790"
format_stamp: "Formatted at 2026-08-07 22:37:11 on dist-test-slave-7kzw"
server_key: "b79d3ffc01472be585058bc1d0f75bb0"
server_key_iv: "a6538627236e45411438fbaa997a3e8a"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:11.262140 13129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "ad4d21b4e2cc4929889396c44e2d9790"
format_stamp: "Formatted at 2026-08-07 22:37:11 on dist-test-slave-7kzw"
server_key: "b79d3ffc01472be585058bc1d0f75bb0"
server_key_iv: "a6538627236e45411438fbaa997a3e8a"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:11.274241 13129 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.000s sys 0.003s
I20260807 22:37:11.289713 13153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:11.290002 13129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:37:11.290108 13129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "ad4d21b4e2cc4929889396c44e2d9790"
format_stamp: "Formatted at 2026-08-07 22:37:11 on dist-test-slave-7kzw"
server_key: "b79d3ffc01472be585058bc1d0f75bb0"
server_key_iv: "a6538627236e45411438fbaa997a3e8a"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:11.290217 13129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:37:11.313917 13129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:37:11.317351 13129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:37:11.317584 13129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:37:11.317955 13129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:37:11.318625 13129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:37:11.318727 13129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:11.318804 13129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:37:11.318866 13129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:11.341063 13129 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:34645
I20260807 22:37:11.341522 13129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:37:11.343546 13315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:34645 every 8 connection(s)
I20260807 22:37:11.345471 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 13129
I20260807 22:37:11.345600 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:37:11.345860 29003 external_mini_cluster.cc:1428] Setting key 9db715d62b6d01cfaf2fa1ebfadd719a
Aug 07 22:37:11 dist-test-slave-7kzw krb5kdc[11999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142231, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:37:11.371003 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.344643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:46215 (local address 127.28.82.254:43823)
0807 22:37:11.344704 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.344706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.344777 (+ 71us) server_negotiation.cc:407] Connection header received
0807 22:37:11.361064 (+ 16287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.361066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.361079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.361124 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:37:11.362049 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.362635 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.363781 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.363997 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.364075 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.367203 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:37:11.367206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:37:11.367206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:37:11.367215 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:37:11.369065 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:11.369592 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:11.369594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:11.369594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:11.369670 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:11.370084 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:11.370085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:11.370085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:11.370231 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:37:11.370335 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.370777 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.370885 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 22:37:11.372018 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.344256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:43823 (local address 127.28.82.195:46215)
0807 22:37:11.344576 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 22:37:11.344634 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 22:37:11.360934 (+ 16300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:37:11.361203 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:37:11.361208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:37:11.361610 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:37:11.361931 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.361937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.362747 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.362748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.363677 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.363679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.364082 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.364084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.364271 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.365053 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:37:11.365062 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:37:11.367090 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:37:11.369196 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:37:11.369203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:37:11.369205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:37:11.369503 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:37:11.369776 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:37:11.369777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:37:11.369777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:37:11.369941 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:37:11.370348 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:37:11.370356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:37:11.370570 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 22:37:11.371499 (+ 929us) client_negotiation.cc:240] Negotiation successful
0807 22:37:11.371712 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":179,"threads_started":1}
I20260807 22:37:11.372766 13316 heartbeater.cc:347] Connected to a master server at 127.28.82.254:43823
I20260807 22:37:11.372905 13316 heartbeater.cc:464] Registering TS with master...
I20260807 22:37:11.373157 13316 heartbeater.cc:511] Master 127.28.82.254:43823 requested a full tablet report, sending...
I20260807 22:37:11.373636 12665 ts_manager.cc:194] Registered new tserver with Master: ad4d21b4e2cc4929889396c44e2d9790 (127.28.82.195:34645)
I20260807 22:37:11.374143 12665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:46215
I20260807 22:37:11.382092 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:37:11.391880 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.384328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.28.82.254:43823)
0807 22:37:11.384372 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.384375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.384713 (+ 338us) server_negotiation.cc:407] Connection header received
0807 22:37:11.384788 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.384789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.384800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.384858 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:37:11.385576 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.386064 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.387117 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.387427 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.387470 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.389041 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:37:11.389043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:37:11.389043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:37:11.389051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:37:11.390627 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:11.391048 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:11.391049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:11.391050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:11.391123 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:11.391336 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:11.391337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:11.391337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:11.391454 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:37:11.391565 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.391696 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.391775 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:37:11.393623 12665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:37:11.395517 12665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:37:11.405735 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.398590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:40119 (local address 127.0.0.1:57116)
0807 22:37:11.399839 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0807 22:37:11.399899 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 22:37:11.400030 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:37:11.400486 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:37:11.400488 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:37:11.400491 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:37:11.400558 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.400559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.402511 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.402512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.403676 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.403679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.405412 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.405413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.405495 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.405512 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 22:37:11.405561 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 22:37:11.405615 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"mutex_wait_us":17,"thread_start_us":602,"threads_started":1}
I20260807 22:37:11.406358 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.399617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:34645 (local address 127.0.0.1:60250)
0807 22:37:11.400745 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0807 22:37:11.400757 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:37:11.400802 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:37:11.401210 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:37:11.401211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:37:11.401213 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:37:11.401351 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.401352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.402781 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.402781 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.404534 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.404536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.405853 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.405854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.406146 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.406151 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:37:11.406218 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 22:37:11.406284 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":1,"thread_start_us":362,"threads_started":1}
I20260807 22:37:11.406505 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.398916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.28.82.194:40119)
0807 22:37:11.399296 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.399311 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.399983 (+ 672us) server_negotiation.cc:407] Connection header received
0807 22:37:11.400209 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.400212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.400231 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.400291 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:37:11.400651 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.402401 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.403922 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.405307 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.406037 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.406268 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.406298 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.406339 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":171,"threads_started":1}
I20260807 22:37:11.407289 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.399805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60250 (local address 127.28.82.195:34645)
0807 22:37:11.401006 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.401011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.401021 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:37:11.401085 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.401087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.401104 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.401431 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:37:11.401555 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.402686 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.404810 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.405750 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.406893 (+ 1143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.407108 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.407139 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.407184 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1143,"thread_start_us":191,"threads_started":1}
I20260807 22:37:11.407406 13040 tablet_service.cc:1467] Processing CreateTablet for tablet 3229ed5dab0341d3806526a60da0617b (DEFAULT_TABLE table=test-table [id=2bd260993b474d90b571ca949426b2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:37:11.407822 13040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3229ed5dab0341d3806526a60da0617b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:37:11.408200 13221 tablet_service.cc:1467] Processing CreateTablet for tablet 3229ed5dab0341d3806526a60da0617b (DEFAULT_TABLE table=test-table [id=2bd260993b474d90b571ca949426b2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:37:11.408564 13221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3229ed5dab0341d3806526a60da0617b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:37:11.414769 13341 tablet_bootstrap.cc:492] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: Bootstrap starting.
I20260807 22:37:11.416181 13341 tablet_bootstrap.cc:654] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: Neither blocks nor log segments found. Creating new log.
I20260807 22:37:11.416909 13341 log.cc:824] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: Log is configured to *not* fsync() on all Append() calls
I20260807 22:37:11.417441 13340 tablet_bootstrap.cc:492] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Bootstrap starting.
I20260807 22:37:11.419068 13341 tablet_bootstrap.cc:492] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: No bootstrap required, opened a new log
I20260807 22:37:11.419147 13341 ts_tablet_manager.cc:1397] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 22:37:11.421772 13341 raft_consensus.cc:348] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.421940 13341 raft_consensus.cc:374] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:37:11.421986 13341 raft_consensus.cc:729] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4d21b4e2cc4929889396c44e2d9790, State: Initialized, Role: FOLLOWER
I20260807 22:37:11.422762 13340 tablet_bootstrap.cc:654] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Neither blocks nor log segments found. Creating new log.
I20260807 22:37:11.423329 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.398816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.28.82.193:42321)
0807 22:37:11.399043 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.399049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.400240 (+ 1191us) server_negotiation.cc:407] Connection header received
0807 22:37:11.400464 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.400468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.400489 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.400578 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:37:11.400971 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.402434 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.421479 (+ 19045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.422754 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.422837 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.422899 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.423202 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.423251 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":168,"threads_started":1}
I20260807 22:37:11.423681 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.398628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42321 (local address 127.0.0.1:37174)
0807 22:37:11.400171 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0807 22:37:11.400182 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:37:11.400271 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:37:11.400671 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:37:11.400672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:37:11.400675 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:37:11.400844 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.400847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.419841 (+ 18994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.419842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.421301 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.421306 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.422940 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.422941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.423067 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.423081 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:37:11.423509 (+ 428us) client_negotiation.cc:240] Negotiation successful
0807 22:37:11.423578 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1510,"thread_start_us":1150,"threads_started":1}
I20260807 22:37:11.423832 13340 log.cc:824] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Log is configured to *not* fsync() on all Append() calls
I20260807 22:37:11.424360 13341 consensus_queue.cc:260] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.424724 13341 ts_tablet_manager.cc:1428] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 22:37:11.424863 13316 heartbeater.cc:503] Master 127.28.82.254:43823 was elected leader, sending a full tablet report...
I20260807 22:37:11.424906 12856 tablet_service.cc:1467] Processing CreateTablet for tablet 3229ed5dab0341d3806526a60da0617b (DEFAULT_TABLE table=test-table [id=2bd260993b474d90b571ca949426b2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:37:11.425287 12856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3229ed5dab0341d3806526a60da0617b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:37:11.427925 13340 tablet_bootstrap.cc:492] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: No bootstrap required, opened a new log
I20260807 22:37:11.428002 13340 ts_tablet_manager.cc:1397] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.002s
I20260807 22:37:11.431411 13348 tablet_bootstrap.cc:492] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Bootstrap starting.
I20260807 22:37:11.432489 13340 raft_consensus.cc:348] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.432762 13340 raft_consensus.cc:374] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:37:11.432813 13340 raft_consensus.cc:729] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17656cf5bcea4822a911021ffbe58e98, State: Initialized, Role: FOLLOWER
I20260807 22:37:11.432964 13340 consensus_queue.cc:260] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.433317 13340 ts_tablet_manager.cc:1428] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260807 22:37:11.433346 13348 tablet_bootstrap.cc:654] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Neither blocks nor log segments found. Creating new log.
I20260807 22:37:11.433459 13123 heartbeater.cc:503] Master 127.28.82.254:43823 was elected leader, sending a full tablet report...
I20260807 22:37:11.434724 13348 log.cc:824] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Log is configured to *not* fsync() on all Append() calls
I20260807 22:37:11.435988 13348 tablet_bootstrap.cc:492] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: No bootstrap required, opened a new log
I20260807 22:37:11.436108 13348 ts_tablet_manager.cc:1397] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 22:37:11.439879 13348 raft_consensus.cc:348] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.440052 13348 raft_consensus.cc:374] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:37:11.440109 13348 raft_consensus.cc:729] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80373d338ec74634b95e00d73a3ffc75, State: Initialized, Role: FOLLOWER
I20260807 22:37:11.440268 13348 consensus_queue.cc:260] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.440601 13348 ts_tablet_manager.cc:1428] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260807 22:37:11.440711 12945 heartbeater.cc:503] Master 127.28.82.254:43823 was elected leader, sending a full tablet report...
W20260807 22:37:11.471225 12946 tablet.cc:2369] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:37:11.531378 13347 raft_consensus.cc:482] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:37:11.531550 13347 raft_consensus.cc:504] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.532137 13347 leader_election.cc:288] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17656cf5bcea4822a911021ffbe58e98 (127.28.82.194:40119), 80373d338ec74634b95e00d73a3ffc75 (127.28.82.193:42321)
I20260807 22:37:11.536958 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.532351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:40119 (local address 127.28.82.195:48741)
0807 22:37:11.532418 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 22:37:11.532431 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:37:11.532481 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:37:11.532994 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:37:11.532995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:37:11.532997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:37:11.533226 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.533230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.534362 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.534363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.535587 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.535590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.536615 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.536617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.536768 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.536772 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:37:11.536832 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 22:37:11.536876 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 22:37:11.537293 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.532715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:48741 (local address 127.28.82.194:40119)
0807 22:37:11.532817 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.532819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.532828 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:37:11.532858 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.532859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.532871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.532935 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:37:11.533327 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.534069 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.535874 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.536489 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.536567 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.536579 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.537179 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.537225 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 22:37:11.537638 13125 tablet.cc:2369] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:37:11.537698 13063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3229ed5dab0341d3806526a60da0617b" candidate_uuid: "ad4d21b4e2cc4929889396c44e2d9790" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17656cf5bcea4822a911021ffbe58e98" is_pre_election: true
I20260807 22:37:11.537881 13063 raft_consensus.cc:2454] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad4d21b4e2cc4929889396c44e2d9790 in term 0.
I20260807 22:37:11.538189 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.532770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:35543 (local address 127.28.82.193:42321)
0807 22:37:11.532905 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.532908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.532917 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:37:11.532947 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.532948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.532957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.533013 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:37:11.533525 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.534430 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.536424 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.537240 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.537296 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.537304 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.538060 (+ 756us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.538110 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 22:37:11.538199 13160 leader_election.cc:302] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17656cf5bcea4822a911021ffbe58e98, ad4d21b4e2cc4929889396c44e2d9790; no voters:
I20260807 22:37:11.538348 13347 raft_consensus.cc:2781] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:37:11.538404 13347 raft_consensus.cc:482] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:37:11.538440 13347 raft_consensus.cc:3037] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:37:11.538795 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.532376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42321 (local address 127.28.82.195:35543)
0807 22:37:11.532710 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 22:37:11.532724 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:37:11.532779 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:37:11.533082 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:37:11.533083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:37:11.533085 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:37:11.533405 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.533409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.534582 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.534583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.536304 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:37:11.536308 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.537688 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.537689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:37:11.537891 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.537897 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:37:11.538402 (+ 505us) client_negotiation.cc:240] Negotiation successful
0807 22:37:11.538530 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":170,"threads_started":1}
I20260807 22:37:11.539968 13347 raft_consensus.cc:504] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.540127 13347 leader_election.cc:288] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [CANDIDATE]: Term 1 election: Requested vote from peers 17656cf5bcea4822a911021ffbe58e98 (127.28.82.194:40119), 80373d338ec74634b95e00d73a3ffc75 (127.28.82.193:42321)
I20260807 22:37:11.540577 12871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3229ed5dab0341d3806526a60da0617b" candidate_uuid: "ad4d21b4e2cc4929889396c44e2d9790" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80373d338ec74634b95e00d73a3ffc75" is_pre_election: true
I20260807 22:37:11.540741 12871 raft_consensus.cc:2454] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad4d21b4e2cc4929889396c44e2d9790 in term 0.
I20260807 22:37:11.540926 12883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3229ed5dab0341d3806526a60da0617b" candidate_uuid: "ad4d21b4e2cc4929889396c44e2d9790" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80373d338ec74634b95e00d73a3ffc75"
I20260807 22:37:11.541035 12883 raft_consensus.cc:3037] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:37:11.542526 12883 raft_consensus.cc:2454] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad4d21b4e2cc4929889396c44e2d9790 in term 1.
I20260807 22:37:11.542863 13156 leader_election.cc:302] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80373d338ec74634b95e00d73a3ffc75, ad4d21b4e2cc4929889396c44e2d9790; no voters:
I20260807 22:37:11.543002 13063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3229ed5dab0341d3806526a60da0617b" candidate_uuid: "ad4d21b4e2cc4929889396c44e2d9790" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17656cf5bcea4822a911021ffbe58e98"
I20260807 22:37:11.543092 13063 raft_consensus.cc:3037] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:37:11.544531 13063 raft_consensus.cc:2454] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad4d21b4e2cc4929889396c44e2d9790 in term 1.
I20260807 22:37:11.544759 13347 raft_consensus.cc:2781] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:37:11.544965 13347 raft_consensus.cc:686] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 1 LEADER]: Becoming Leader. State: Replica: ad4d21b4e2cc4929889396c44e2d9790, State: Running, Role: LEADER
I20260807 22:37:11.545070 13347 consensus_queue.cc:237] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } }
I20260807 22:37:11.546026 12665 catalog_manager.cc:5495] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad4d21b4e2cc4929889396c44e2d9790 (127.28.82.195). New cstate: current_term: 1 leader_uuid: "ad4d21b4e2cc4929889396c44e2d9790" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad4d21b4e2cc4929889396c44e2d9790" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 34645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 } health_report { overall_health: UNKNOWN } } }
I20260807 22:37:11.558277 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:11.554435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60258 (local address 127.28.82.195:34645)
0807 22:37:11.554629 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:11.554631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:11.554637 (+ 6us) server_negotiation.cc:407] Connection header received
0807 22:37:11.554663 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:11.554663 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:11.554679 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:11.554761 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:37:11.555066 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.556020 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.557198 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:11.557489 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:11.557552 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:11.557870 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:37:11.557993 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:37:11.558100 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:11.558181 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 22:37:11.558205 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260807 22:37:11.560807 12883 raft_consensus.cc:1261] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 1 FOLLOWER]: Refusing update from remote peer ad4d21b4e2cc4929889396c44e2d9790: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:37:11.560868 13063 raft_consensus.cc:1261] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 1 FOLLOWER]: Refusing update from remote peer ad4d21b4e2cc4929889396c44e2d9790: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:37:11.561251 13355 consensus_queue.cc:1035] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80373d338ec74634b95e00d73a3ffc75" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:37:11.561393 13347 consensus_queue.cc:1035] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17656cf5bcea4822a911021ffbe58e98" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:37:11.564926 13362 mvcc.cc:205] Tried to move back new op lower bound from 7316038580470157312 to 7316038580409380864. Current Snapshot: MvccSnapshot[applied={T|T < 7316038580470157312}]
I20260807 22:37:11.568478 12665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:37:11.568583 12665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:37:11.569335 12665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e2adfa8a35d42ac8dbf7c47bc840638: Sending DeleteTablet for 3 replicas of tablet 3229ed5dab0341d3806526a60da0617b
I20260807 22:37:11.569842 12856 tablet_service.cc:1514] Processing DeleteTablet for tablet 3229ed5dab0341d3806526a60da0617b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:37:11 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:37174
I20260807 22:37:11.570319 13373 tablet_replica.cc:330] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: stopping tablet replica
I20260807 22:37:11.570541 13373 raft_consensus.cc:2229] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:37:11.570713 13373 pending_rounds.cc:62] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Trying to abort 2 pending ops.
I20260807 22:37:11.570811 13373 pending_rounds.cc:68] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316038580470157312 op_type: WRITE_OP write_request { tablet_id: "3229ed5dab0341d3806526a60da0617b" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\303\276\331\323\006\"8\n\ntest-admin\022*\n 2bd260993b474d90b571ca949426b2b3\020\001\030\001 \001(\001" signature: "<redacted>""\260\355\313fn\236\241\332\200\271h\337P\217R+c\3350\346;\260hk\2225O\277$\367\027\360\267\352\235\350\027e\357\005\265!\376J\337\227\177\350\341\246\224^T\327\243A\332\\\324\353\270;\255\221" signing_key_seq_num: 0 } } request_id { client_id: "a24d7a828aa1477d81dfcd8877bd3770" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260807 22:37:11.570931 13366 mvcc.cc:205] Tried to move back new op lower bound from 7316038580470157312 to 7316038580409380864. Current Snapshot: MvccSnapshot[applied={T|T < 7316038580470157312}]
I20260807 22:37:11.571199 13373 pending_rounds.cc:68] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316038580409380864 op_type: NO_OP noop_request { }
I20260807 22:37:11.571328 13373 raft_consensus.cc:2866] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260807 22:37:11.571606 13373 raft_consensus.cc:2258] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 22:37:11.571674 13367 consensus_queue.cc:1159] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "80373d338ec74634b95e00d73a3ffc75" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
W20260807 22:37:11.572769 13156 consensus_peers.cc:487] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 -> Peer 80373d338ec74634b95e00d73a3ffc75 (127.28.82.193:42321): Couldn't send request to peer 80373d338ec74634b95e00d73a3ffc75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260807 22:37:11.573012 13373 ts_tablet_manager.cc:1905] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:37:11.576814 13373 ts_tablet_manager.cc:1918] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:37:11.576963 13373 log.cc:1196] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/3229ed5dab0341d3806526a60da0617b
I20260807 22:37:11.577301 13373 ts_tablet_manager.cc:1939] T 3229ed5dab0341d3806526a60da0617b P 80373d338ec74634b95e00d73a3ffc75: Deleting consensus metadata
I20260807 22:37:11.577924 12643 catalog_manager.cc:4841] TS 80373d338ec74634b95e00d73a3ffc75 (127.28.82.193:42321): tablet 3229ed5dab0341d3806526a60da0617b (table test-table [id=2bd260993b474d90b571ca949426b2b3]) successfully deleted
I20260807 22:37:11.570415 13040 tablet_service.cc:1514] Processing DeleteTablet for tablet 3229ed5dab0341d3806526a60da0617b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:37:11 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:57116
I20260807 22:37:11.578732 13221 tablet_service.cc:1514] Processing DeleteTablet for tablet 3229ed5dab0341d3806526a60da0617b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:37:11 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:60250
I20260807 22:37:11.579082 13378 tablet_replica.cc:330] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: stopping tablet replica
I20260807 22:37:11.579219 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 12768
I20260807 22:37:11.579253 13378 raft_consensus.cc:2229] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:37:11.579449 13378 raft_consensus.cc:2258] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:37:11.583123 13377 tablet_replica.cc:330] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: stopping tablet replica
I20260807 22:37:11.583240 13377 raft_consensus.cc:2229] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:37:11.583356 13377 raft_consensus.cc:2258] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:37:11.585409 13377 ts_tablet_manager.cc:1905] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:37:11.587980 13377 ts_tablet_manager.cc:1918] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:37:11.588137 13377 log.cc:1196] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/3229ed5dab0341d3806526a60da0617b
I20260807 22:37:11.588624 13377 ts_tablet_manager.cc:1939] T 3229ed5dab0341d3806526a60da0617b P 17656cf5bcea4822a911021ffbe58e98: Deleting consensus metadata
I20260807 22:37:11.589998 12645 catalog_manager.cc:4841] TS 17656cf5bcea4822a911021ffbe58e98 (127.28.82.194:40119): tablet 3229ed5dab0341d3806526a60da0617b (table test-table [id=2bd260993b474d90b571ca949426b2b3]) successfully deleted
I20260807 22:37:11.596750 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 12953
I20260807 22:37:11.600447 13378 ts_tablet_manager.cc:1905] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:37:11.602190 13378 ts_tablet_manager.cc:1918] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:37:11.602259 13378 log.cc:1196] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/3229ed5dab0341d3806526a60da0617b
I20260807 22:37:11.602562 13378 ts_tablet_manager.cc:1939] T 3229ed5dab0341d3806526a60da0617b P ad4d21b4e2cc4929889396c44e2d9790: Deleting consensus metadata
I20260807 22:37:11.603024 12645 catalog_manager.cc:4841] TS ad4d21b4e2cc4929889396c44e2d9790 (127.28.82.195:34645): tablet 3229ed5dab0341d3806526a60da0617b (table test-table [id=2bd260993b474d90b571ca949426b2b3]) successfully deleted
I20260807 22:37:11.609637 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 13129
I20260807 22:37:11.621035 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 12621
I20260807 22:37:11.634418 29003 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 22:37:11.979430 29003 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 22:37:11.979571 29003 mini_ranger.cc:66] Stopping Ranger...
I20260807 22:37:12.344424 29003 mini_ranger.cc:68] Stopped Ranger
2026-08-07 22:37:12.344 UTC [12040] LOG: received smart shutdown request
2026-08-07 22:37:12.349 UTC [12040] LOG: background worker "logical replication launcher" (PID 12053) exited with exit code 1
2026-08-07 22:37:12.350 UTC [12045] LOG: shutting down
2026-08-07 22:37:12.553 UTC [12040] LOG: database system is shut down
2026-08-07T22:37:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50610 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:37:12 dist-test-slave-7kzw krb5kdc[13392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:37:12 dist-test-slave-7kzw krb5kdc[13392](info): set up 2 sockets
Aug 07 22:37:12 dist-test-slave-7kzw krb5kdc[13392](info): commencing operation
krb5kdc: starting...
W20260807 22:37:14.775204 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.981s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:37:14 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:37:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:37:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.82.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.82.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/rangeradmin_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/rangeradmin_127.28.82.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.82.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.82.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/rangerlookup_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/rangerlookup_127.28.82.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.82.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.82.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/HTTP_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/HTTP_127.28.82.212@KRBTEST.COM.keytab.
I20260807 22:37:15.042186 29003 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskj8JEAI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/postgres -l logfile start
2026-08-07 22:37:17.782 UTC [13435] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 22:37:17.782 UTC [13435] LOG: listening on IPv4 address "127.28.82.212", port 44403
2026-08-07 22:37:17.786 UTC [13435] LOG: listening on Unix socket "/tmp/.s.PGSQL.44403"
2026-08-07 22:37:17.808 UTC [13439] LOG: database system was shut down at 2026-08-07 22:37:16 UTC
2026-08-07 22:37:17.812 UTC [13435] LOG: database system is ready to accept connections
I20260807 22:37:18.942792 29003 mini_postgres.cc:96] Postgres bound to 44403
2026-08-07 22:37:18.950 UTC [13450] FATAL: database "slave" does not exist
127.28.82.212:44403 - accepting connections
I20260807 22:37:18.951117 29003 mini_ranger.cc:161] Starting Ranger...
I20260807 22:37:18.967105 29003 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 22:37:19.495642 29003 mini_ranger.cc:87] Created ranger Postgres database
I20260807 22:37:19.495762 29003 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-admin
2026-08-07 22:37:19,779 [I] DB FLAVOR :POSTGRES
2026-08-07 22:37:19,780 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:37:19,780 [I] Checking connection..
2026-08-07 22:37:19,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 22:37:20,050 [I] Checking connection passed.
2026-08-07 22:37:20,051 [I] --------- Verifying version history table ---------
2026-08-07 22:37:20,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:37:20,302 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:37:20,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:37:20,601 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 22:37:20,602 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 22:37:20,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 22:37:20.879 UTC [13542] WARNING: there is no transaction in progress
2026-08-07 22:37:20,886 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 22:37:20,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 22:37:21,148 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 22:37:21,149 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:37:21,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 22:37:21,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 22:37:21,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7kzw.c.gcp-upstream.internal','N') ;"
2026-08-07 22:37:21,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:37:22,186 [I] Table x_portal_user does not exist in database ranger
2026-08-07 22:37:22,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:37:22,443 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 22:37:22,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:37:22,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:37:22,958 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 22:37:22,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 22:37:23.968 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:23.985 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.004 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.020 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.041 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.059 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.115 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.122 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.135 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.146 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.155 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.171 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.181 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.188 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.196 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24.204 UTC [13699] WARNING: there is no transaction in progress
2026-08-07 22:37:24,930 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 22:37:24,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 22:37:25,183 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 22:37:25,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 22:37:25,438 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 22:37:25,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 22:37:25,697 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 22:37:25,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 22:37:25,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 22:37:26,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7kzw.c.gcp-upstream.internal';"
2026-08-07 22:37:26,510 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 22:37:26,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 22:37:26,785 [I] DB_PATCHES have already been applied
I20260807 22:37:26.791965 29003 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-admin:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskj8JEAI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 22:37:26.792086 29003 mini_ranger.cc:193] Using host: 127.28.82.212
I20260807 22:37:26.796641 29003 mini_ranger.cc:227] Ranger admin URL: http://127.28.82.212:34785
Aug 07, 2026 10:37:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:37:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:37:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:37:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:37:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:37:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:37:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 22:37:28.127718 29003 mini_ranger.cc:160] Time spent starting Ranger: real 9.177s user 0.002s sys 0.010s
Aug 07, 2026 10:37:49 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:37:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 10:37:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:37:49 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:37:50 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 22:37:50.620280 29003 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.82.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.82.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/rangerkms_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.82.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/rangerkms_127.28.82.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.82.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.82.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.82.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/HTTP_127.28.82.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.82.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/HTTP_127.28.82.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 22:37:50 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142270, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 22:37:50.716305 29003 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 22:37:50.728206 29003 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 22:37:51.565613 29003 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 22:37:51.565708 29003 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms
I20260807 22:37:51.565730 29003 mini_ranger_kms.cc:227] Using postgres at 127.28.82.212:44403
2026-08-07 22:37:51,785 [I] DB FLAVOR :POSTGRES
2026-08-07 22:37:51,786 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 22:37:51,786 [I] Checking connection
2026-08-07 22:37:51,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 22:37:52,093 [I] connection success
2026-08-07 22:37:52,093 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 22:37:52,093 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 22:37:52,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 22:37:52,326 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 22:37:52,326 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 22:37:52,327 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 22:37:52,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.82.212:44403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 22:37:52,603 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 22:37:53.109477 29003 mini_ranger_kms.cc:310] Created kms service
I20260807 22:37:53.316772 29003 mini_ranger_kms.cc:326] Created kudu user
I20260807 22:37:53.359190 29003 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 22:37:53 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142270, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:37:53.810636 29003 mini_ranger_kms.cc:384] Added ranger policy
I20260807 22:37:53.810853 29003 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms:/tmp/dist-test-taskj8JEAI/build/release/bin/postgresql.jar:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskj8JEAI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 22:37:53.810874 29003 mini_ranger_kms.cc:240] Using host: 127.28.82.212
I20260807 22:37:53.824033 29003 mini_ranger_kms.cc:276] Ranger KMS PID: 13953
I20260807 22:37:53.824123 29003 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.82.212:44181
Aug 07, 2026 10:37:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:37:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:37:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:37:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:37:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:37:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 22:37:56 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142276, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.82.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:37:56.515189 29003 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.799s user 0.016s sys 0.187s
I20260807 22:37:56.515298 29003 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 22:37:56.515342 29003 mini_ranger_kms.cc:401] 127.28.82.212:44181/kms/v1/keys
Aug 07 22:37:56 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142276, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.82.212@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
Aug 07 22:37:57 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142276, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.82.212@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
Aug 07 22:37:59 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:37:59.466840 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:36881
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:36881
--ranger_config_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:44181/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:37:59.553058 14026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:37:59.553269 14026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:37:59.553292 14026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:37:59.554494 14026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:37:59.554569 14026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:37:59.554593 14026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:37:59.554615 14026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:37:59.554668 14026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:37:59.556248 14026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.82.212:44181/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:36881
--ranger_config_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:37:59.556546 14026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:37:59.556795 14026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:37:59.562820 14036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:37:59.562809 14034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:37:59.562809 14033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:37:59.563032 14026 server_base.cc:1034] running on GCE node
I20260807 22:37:59.563419 14026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:37:59.563735 14026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:37:59.564914 14026 hybrid_clock.cc:648] HybridClock initialized: now 1786142279564888 us; error 43 us; skew 500 ppm
Aug 07 22:37:59 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:37:59.567656 14026 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:37:59.568158 14026 webserver.cc:460] Webserver started at http://127.28.82.254:34181/ using document root <none> and password file <none>
I20260807 22:37:59.568395 14026 fs_manager.cc:359] Metadata directory not provided
I20260807 22:37:59.568459 14026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:37:59.568627 14026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:37:59 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:37:59.629274 14026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "937a24ad67d748d09ead70092c9ac4a8"
format_stamp: "Formatted at 2026-08-07 22:37:59 on dist-test-slave-7kzw"
server_key: "88974e4b2a395cf6af6416b4ff606983"
server_key_iv: "359408b6573bd639250eb6cb470008bb"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:59.629721 14026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "937a24ad67d748d09ead70092c9ac4a8"
format_stamp: "Formatted at 2026-08-07 22:37:59 on dist-test-slave-7kzw"
server_key: "88974e4b2a395cf6af6416b4ff606983"
server_key_iv: "359408b6573bd639250eb6cb470008bb"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:59.631321 14026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 22:37:59.645854 14043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:37:59.646082 14026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:37:59.646193 14026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "937a24ad67d748d09ead70092c9ac4a8"
format_stamp: "Formatted at 2026-08-07 22:37:59 on dist-test-slave-7kzw"
server_key: "88974e4b2a395cf6af6416b4ff606983"
server_key_iv: "359408b6573bd639250eb6cb470008bb"
server_key_version: "kuduclusterkey@0"
I20260807 22:37:59.646294 14026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:37:59.661278 14026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:37:59.664111 14026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:37:59.664291 14026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:37:59.669898 14026 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:36881
I20260807 22:37:59.670532 14026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:37:59.670511 14118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:36881 every 8 connection(s)
I20260807 22:37:59.671357 14119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:37:59.673671 14119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Bootstrap starting.
I20260807 22:37:59.673993 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 14026
I20260807 22:37:59.674115 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:37:59.674310 29003 external_mini_cluster.cc:1428] Setting key a2bd6461001376dc854e3c9ed54a43a9
I20260807 22:37:59.675071 14119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Neither blocks nor log segments found. Creating new log.
I20260807 22:37:59.675436 14119 log.cc:824] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:37:59.676275 14119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: No bootstrap required, opened a new log
I20260807 22:37:59.678210 14119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } }
I20260807 22:37:59.678380 14119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:37:59.678413 14119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937a24ad67d748d09ead70092c9ac4a8, State: Initialized, Role: FOLLOWER
I20260807 22:37:59.678606 14119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } }
I20260807 22:37:59.678697 14119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:37:59.678725 14119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:37:59.678790 14119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:37:59.679338 14119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } }
I20260807 22:37:59.679467 14119 leader_election.cc:302] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 937a24ad67d748d09ead70092c9ac4a8; no voters:
I20260807 22:37:59.679653 14119 leader_election.cc:288] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:37:59.680015 14119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:37:59.679692 14124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:37:59.681154 14124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 1 LEADER]: Becoming Leader. State: Replica: 937a24ad67d748d09ead70092c9ac4a8, State: Running, Role: LEADER
I20260807 22:37:59.681334 14124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } }
I20260807 22:37:59.681682 14124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 937a24ad67d748d09ead70092c9ac4a8. Latest consensus state: current_term: 1 leader_uuid: "937a24ad67d748d09ead70092c9ac4a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } } }
I20260807 22:37:59.681862 14124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:37:59.682117 14140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 22:37:59 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:37:59.682896 14119 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:37:59.683051 14125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "937a24ad67d748d09ead70092c9ac4a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } } }
I20260807 22:37:59.683852 14125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:37:59.686785 14140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:37:59.691941 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:37:59.674860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.28.82.254:36881)
0807 22:37:59.675059 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 22:37:59.675069 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:37:59.675268 (+ 199us) server_negotiation.cc:407] Connection header received
0807 22:37:59.675829 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:37:59.675837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:37:59.675895 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:37:59.676063 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:37:59.676844 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:59.678804 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:59.679805 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:37:59.680423 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:37:59.680546 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:37:59.683511 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:37:59.683515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:37:59.683515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:37:59.683528 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:37:59.689149 (+ 5621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:59.689663 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:59.689667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:59.689667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:59.689852 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:37:59.690122 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:37:59.690123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:37:59.690123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:37:59.690647 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:37:59.691292 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0807 22:37:59.691373 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 22:37:59.691650 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260807 22:37:59.693876 14140 catalog_manager.cc:1347] Generated new cluster ID: faac878b15634dd4b8dc08bd66d0417e
I20260807 22:37:59.693928 14140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:37:59.720510 14140 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:37:59.721346 14140 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:37:59.726413 14140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Generated new TSK 0
I20260807 22:37:59.726696 14140 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:38:00.527098 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:44181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36881
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:00.662968 14176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:00.663261 14176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:00.663304 14176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:00.665006 14176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:00.665104 14176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:00.665167 14176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:38:00.667408 14176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.82.212:44181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:00.667671 14176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:00.668008 14176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:00.677321 14187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:00.677820 14176 server_base.cc:1034] running on GCE node
W20260807 22:38:00.677773 14191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:00.678118 14188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:00.678365 14176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:00.678684 14176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:00.679858 14176 hybrid_clock.cc:648] HybridClock initialized: now 1786142280679835 us; error 35 us; skew 500 ppm
Aug 07 22:38:00 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142280, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:00.683346 14176 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:38:00.683847 14176 webserver.cc:460] Webserver started at http://127.28.82.193:43951/ using document root <none> and password file <none>
I20260807 22:38:00.684135 14176 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:00.684199 14176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:00.684713 14176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:38:00 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142280, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:38:00.699927 14176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "a26784e1ec604981a80330b93e625c2b"
format_stamp: "Formatted at 2026-08-07 22:38:00 on dist-test-slave-7kzw"
server_key: "c523a4e80ff20033d8e644375f540188"
server_key_iv: "854778dfc9c7fb2c08996d9c790b0d20"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:00.700480 14176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "a26784e1ec604981a80330b93e625c2b"
format_stamp: "Formatted at 2026-08-07 22:38:00 on dist-test-slave-7kzw"
server_key: "c523a4e80ff20033d8e644375f540188"
server_key_iv: "854778dfc9c7fb2c08996d9c790b0d20"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:00.702077 14176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:00.721696 14203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:00.722069 14176 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260807 22:38:00.722255 14176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "a26784e1ec604981a80330b93e625c2b"
format_stamp: "Formatted at 2026-08-07 22:38:00 on dist-test-slave-7kzw"
server_key: "c523a4e80ff20033d8e644375f540188"
server_key_iv: "854778dfc9c7fb2c08996d9c790b0d20"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:00.722430 14176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:00.745817 14176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:00.748658 14176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:00.748904 14176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:00.749301 14176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:00.749788 14176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:00.749899 14176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:00.750036 14176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:00.750128 14176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:00.758037 14176 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:38509
I20260807 22:38:00.760759 14363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:38509 every 8 connection(s)
I20260807 22:38:00.761089 14176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:38:00.767050 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 14176
I20260807 22:38:00.767158 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:38:00.767439 29003 external_mini_cluster.cc:1428] Setting key ef098ec225d82a19f2cc6e1d757e2ba2
Aug 07 22:38:00 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142280, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:00.772670 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:00.762209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:51045 (local address 127.28.82.254:36881)
0807 22:38:00.762353 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:00.762355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:00.762609 (+ 254us) server_negotiation.cc:407] Connection header received
0807 22:38:00.763364 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:00.763366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:00.763379 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:00.763447 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:00.764181 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:00.764673 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:00.766154 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:00.766338 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:00.766389 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:00.769238 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:00.769240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:00.769240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:00.769249 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:00.771049 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:00.771584 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:00.771585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:00.771585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:00.771655 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:00.771948 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:00.771949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:00.771949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:00.772086 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:00.772182 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:00.772478 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 22:38:00.772571 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260807 22:38:00.772850 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:00.762146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36881 (local address 127.28.82.193:51045)
0807 22:38:00.762435 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:00.762510 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:00.763227 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:00.763525 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:00.763529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:00.763787 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:00.764080 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:00.764088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:00.764780 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:00.764782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:00.766056 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:00.766060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:00.766447 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:00.766449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:00.766651 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:00.767332 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:00.767343 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:00.769122 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:00.771158 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:00.771165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:00.771167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:00.771491 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:00.771732 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:00.771733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:00.771734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:00.771865 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:00.772177 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:00.772185 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:00.772331 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 22:38:00.772443 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:38:00.772589 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":94,"threads_started":1}
I20260807 22:38:00.773458 14364 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36881
I20260807 22:38:00.773581 14364 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:00.773768 14364 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:00.774343 14065 ts_manager.cc:194] Registered new tserver with Master: a26784e1ec604981a80330b93e625c2b (127.28.82.193:38509)
I20260807 22:38:00.774979 14065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:51045
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:38:00.803543 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:44181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36881
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:38:00 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142280, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 22:38:00.930523 14376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:00.930902 14376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:00.930985 14376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:00.932793 14376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:00.932862 14376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:00.932901 14376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:38:00.935154 14376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.82.212:44181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:00.935631 14376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:00.935985 14376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:00.942371 14387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:00.942437 14384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:00.942849 14385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:00.943104 14376 server_base.cc:1034] running on GCE node
I20260807 22:38:00.943323 14376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:00.943598 14376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:00.944753 14376 hybrid_clock.cc:648] HybridClock initialized: now 1786142280944736 us; error 33 us; skew 500 ppm
Aug 07 22:38:00 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142280, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:00.948345 14376 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:38:00.948868 14376 webserver.cc:460] Webserver started at http://127.28.82.194:37163/ using document root <none> and password file <none>
I20260807 22:38:00.949193 14376 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:00.949256 14376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:00.949406 14376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:38:00 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142280, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:38:00.972499 14376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "41657cc2387545e1b18556ecfe392c00"
format_stamp: "Formatted at 2026-08-07 22:38:00 on dist-test-slave-7kzw"
server_key: "8db8c194edf1cf364393526c478bfefb"
server_key_iv: "1035f39e6cd9b9e1cc46f6f8990c0450"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:00.973100 14376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "41657cc2387545e1b18556ecfe392c00"
format_stamp: "Formatted at 2026-08-07 22:38:00 on dist-test-slave-7kzw"
server_key: "8db8c194edf1cf364393526c478bfefb"
server_key_iv: "1035f39e6cd9b9e1cc46f6f8990c0450"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:00.975049 14376 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:00.988773 14398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:00.989004 14376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:00.989090 14376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "41657cc2387545e1b18556ecfe392c00"
format_stamp: "Formatted at 2026-08-07 22:38:00 on dist-test-slave-7kzw"
server_key: "8db8c194edf1cf364393526c478bfefb"
server_key_iv: "1035f39e6cd9b9e1cc46f6f8990c0450"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:00.989179 14376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:01.034803 14376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:01.037415 14376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:01.037626 14376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:01.037936 14376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:01.038403 14376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:01.038520 14376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.038600 14376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:01.038659 14376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.045382 14376 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:41215
I20260807 22:38:01.045871 14376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:38:01.048655 14557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:41215 every 8 connection(s)
I20260807 22:38:01.052480 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 14376
I20260807 22:38:01.052589 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:38:01.052821 29003 external_mini_cluster.cc:1428] Setting key a792ebbec7dbe51c69b978466da1d4d1
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142280, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:01.061635 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:01.049648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:58541 (local address 127.28.82.254:36881)
0807 22:38:01.049709 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:01.049712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:01.050008 (+ 296us) server_negotiation.cc:407] Connection header received
0807 22:38:01.050771 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:01.050772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:01.050788 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:01.050820 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:01.051695 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.052462 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.054912 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.055230 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.055274 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:01.058049 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:01.058051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:01.058051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:01.058059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:01.059903 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:01.060442 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:01.060444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:01.060444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:01.060509 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:01.060818 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:01.060819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:01.060820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:01.060960 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:01.061009 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:01.061410 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 22:38:01.061504 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 22:38:01.062399 14563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:01.049329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36881 (local address 127.28.82.194:58541)
0807 22:38:01.049874 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:01.049921 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:01.050630 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:01.051006 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:01.051010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:01.051283 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:01.051572 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:01.051579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.052948 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.052949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:01.054807 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:01.054811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.055378 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.055379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:01.055536 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:01.056149 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:01.056173 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:01.057925 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:01.060017 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:01.060024 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:01.060027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:01.060342 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:01.060589 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:01.060590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:01.060590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:01.060730 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:01.061065 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:01.061071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:01.061239 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 22:38:01.061981 (+ 742us) client_negotiation.cc:240] Negotiation successful
0807 22:38:01.062147 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":119,"threads_started":1}
I20260807 22:38:01.063205 14558 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36881
I20260807 22:38:01.063303 14558 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:01.063498 14558 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:01.063848 14065 ts_manager.cc:194] Registered new tserver with Master: 41657cc2387545e1b18556ecfe392c00 (127.28.82.194:41215)
I20260807 22:38:01.064313 14065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:58541
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:38:01.105808 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:44181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36881
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:01.218562 14566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:01.218806 14566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:01.218850 14566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:01.220028 14566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:01.220108 14566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:01.220155 14566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:38:01.221707 14566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.82.212:44181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:01.221980 14566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:01.222258 14566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:01.228919 14577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:01.229120 14580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:01.229569 14566 server_base.cc:1034] running on GCE node
W20260807 22:38:01.229781 14574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:01.230088 14566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:01.230387 14566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:01.231544 14566 hybrid_clock.cc:648] HybridClock initialized: now 1786142281231535 us; error 51 us; skew 500 ppm
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:01.234268 14566 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:38:01.234791 14566 webserver.cc:460] Webserver started at http://127.28.82.195:45299/ using document root <none> and password file <none>
I20260807 22:38:01.235034 14566 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:01.235095 14566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:01.235227 14566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:38:01.251063 14566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "38f10e4fb3d54405b27192988ae51979"
format_stamp: "Formatted at 2026-08-07 22:38:01 on dist-test-slave-7kzw"
server_key: "5768ad2c0bb1dc0c8bf5eff53434c350"
server_key_iv: "6ee5f3acbab3be6e6adec5fa17508fd4"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:01.251528 14566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "38f10e4fb3d54405b27192988ae51979"
format_stamp: "Formatted at 2026-08-07 22:38:01 on dist-test-slave-7kzw"
server_key: "5768ad2c0bb1dc0c8bf5eff53434c350"
server_key_iv: "6ee5f3acbab3be6e6adec5fa17508fd4"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:01.253471 14566 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:01.279654 14589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.280099 14566 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260807 22:38:01.280225 14566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "38f10e4fb3d54405b27192988ae51979"
format_stamp: "Formatted at 2026-08-07 22:38:01 on dist-test-slave-7kzw"
server_key: "5768ad2c0bb1dc0c8bf5eff53434c350"
server_key_iv: "6ee5f3acbab3be6e6adec5fa17508fd4"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:01.280339 14566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:01.302081 14566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:01.304603 14566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:01.304840 14566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:01.305181 14566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:01.305675 14566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:01.305778 14566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.305860 14566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:01.305923 14566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.313534 14566 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:35799
I20260807 22:38:01.313812 14754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:35799 every 8 connection(s)
I20260807 22:38:01.314154 14566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:38:01.319263 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 14566
I20260807 22:38:01.319376 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:38:01.319578 29003 external_mini_cluster.cc:1428] Setting key 7d428706219bf626a1dfc5df1e1ee97a
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:01.326645 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:01.315238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:56709 (local address 127.28.82.254:36881)
0807 22:38:01.315285 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:01.315287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:01.315482 (+ 195us) server_negotiation.cc:407] Connection header received
0807 22:38:01.316186 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:01.316188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:01.316201 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:01.316233 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:01.317081 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.317863 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.319470 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.319771 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.319813 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:01.323122 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:01.323123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:01.323124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:01.323132 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:01.324929 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:01.325446 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:01.325447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:01.325448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:01.325514 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:01.325850 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:01.325851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:01.325852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:01.325990 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:01.326036 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:01.326427 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 22:38:01.326537 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:38:01.327404 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:01.314939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36881 (local address 127.28.82.195:56709)
0807 22:38:01.315358 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:01.315402 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:01.316054 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:01.316416 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:01.316421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:01.316680 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:01.316956 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:01.316961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.317975 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.317976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:01.319269 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:01.319272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.320573 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.320575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:01.320718 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:01.321374 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:01.321385 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:01.323007 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:01.325033 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:01.325040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:01.325043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:01.325345 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:01.325594 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:01.325595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:01.325595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:01.325759 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:01.326086 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:01.326092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:01.326256 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 22:38:01.326970 (+ 714us) client_negotiation.cc:240] Negotiation successful
0807 22:38:01.327140 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":102,"threads_started":1}
I20260807 22:38:01.328142 14756 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36881
I20260807 22:38:01.328241 14756 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:01.328450 14756 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:01.328893 14065 ts_manager.cc:194] Registered new tserver with Master: 38f10e4fb3d54405b27192988ae51979 (127.28.82.195:35799)
I20260807 22:38:01.329370 14065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:56709
I20260807 22:38:01.331254 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:38:01.332151 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 14176
I20260807 22:38:01.344415 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 14376
I20260807 22:38:01.351451 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 14566
I20260807 22:38:01.359874 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 14026
I20260807 22:38:01.373111 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:36881
--webserver_interface=127.28.82.254
--webserver_port=34181
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:36881
--ranger_config_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:44181/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.82.212:44181/kms with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:01.455610 14761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:01.455818 14761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:01.455839 14761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:01.456972 14761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:01.457016 14761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:01.457031 14761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:01.457082 14761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:01.457103 14761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:01.458837 14761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.82.212:44181/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:36881
--ranger_config_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=34181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:01.459084 14761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:01.459333 14761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:01.464836 14767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:01.464828 14772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:01.464885 14768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:01.465142 14761 server_base.cc:1034] running on GCE node
I20260807 22:38:01.465381 14761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:01.465613 14761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:01.466765 14761 hybrid_clock.cc:648] HybridClock initialized: now 1786142281466754 us; error 29 us; skew 500 ppm
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:01.469434 14761 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:38:01.469933 14761 webserver.cc:460] Webserver started at http://127.28.82.254:34181/ using document root <none> and password file <none>
I20260807 22:38:01.470175 14761 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:01.470229 14761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:38:01.510819 14761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:01.511739 14783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.512003 14761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:01.512123 14761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "937a24ad67d748d09ead70092c9ac4a8"
format_stamp: "Formatted at 2026-08-07 22:37:59 on dist-test-slave-7kzw"
server_key: "88974e4b2a395cf6af6416b4ff606983"
server_key_iv: "359408b6573bd639250eb6cb470008bb"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:01.512492 14761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:01.533123 14761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:01.535367 14761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:01.535544 14761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:01.540011 14761 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:36881
I20260807 22:38:01.540115 14862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:36881 every 8 connection(s)
I20260807 22:38:01.540498 14761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:38:01.541795 14863 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:38:01.542529 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Bootstrap starting.
I20260807 22:38:01.549402 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 14761
I20260807 22:38:01.549767 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:44181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:38509
--local_ip_for_outbound_sockets=127.28.82.193
--tserver_master_addrs=127.28.82.254:36881
--webserver_port=43951
--webserver_interface=127.28.82.193
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.82.212:44181/kms with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:01.550783 14863 log.cc:824] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:01.553165 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:38:01.553468 14863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Bootstrap complete.
I20260807 22:38:01.557580 14863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } }
I20260807 22:38:01.557996 14863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937a24ad67d748d09ead70092c9ac4a8, State: Initialized, Role: FOLLOWER
I20260807 22:38:01.558204 14863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } }
I20260807 22:38:01.558298 14863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:38:01.558357 14863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:38:01.558415 14863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:38:01.560050 14863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } }
I20260807 22:38:01.560194 14863 leader_election.cc:302] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 937a24ad67d748d09ead70092c9ac4a8; no voters:
I20260807 22:38:01.560384 14863 leader_election.cc:288] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:38:01.560463 14869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:38:01.560663 14869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [term 2 LEADER]: Becoming Leader. State: Replica: 937a24ad67d748d09ead70092c9ac4a8, State: Running, Role: LEADER
I20260807 22:38:01.560680 14863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:01.560874 14869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } }
I20260807 22:38:01.561234 14871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "937a24ad67d748d09ead70092c9ac4a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } } }
I20260807 22:38:01.561233 14872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 937a24ad67d748d09ead70092c9ac4a8. Latest consensus state: current_term: 2 leader_uuid: "937a24ad67d748d09ead70092c9ac4a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "937a24ad67d748d09ead70092c9ac4a8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36881 } } }
I20260807 22:38:01.561350 14871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:01.561357 14872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:01.561867 14878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:38:01.562597 14878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:38:01.562789 14878 catalog_manager.cc:1259] Loaded cluster ID: faac878b15634dd4b8dc08bd66d0417e
I20260807 22:38:01.562873 14878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:38:01.563001 14863 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 22:38:01.568337 14878 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:38:01.568892 14878 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Loaded TSK: 0
I20260807 22:38:01.569206 14878 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:38:01.643719 14868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:01.643958 14868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:01.643999 14868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:01.645741 14868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:01.645807 14868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:01.645849 14868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:38:01.648114 14868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.82.212:44181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:38509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=43951
--tserver_master_addrs=127.28.82.254:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:01.648370 14868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:01.648654 14868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:01.655732 14915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:01.655732 14917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:01.655743 14914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:01.656441 14868 server_base.cc:1034] running on GCE node
I20260807 22:38:01.656668 14868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:01.656944 14868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:01.660251 14868 hybrid_clock.cc:648] HybridClock initialized: now 1786142281660210 us; error 50 us; skew 500 ppm
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:01.663573 14868 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:38:01.664110 14868 webserver.cc:460] Webserver started at http://127.28.82.193:43951/ using document root <none> and password file <none>
I20260807 22:38:01.664366 14868 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:01.664436 14868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:38:01.691120 14868 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:38:01.692292 14929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.692528 14868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:38:01.692665 14868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "a26784e1ec604981a80330b93e625c2b"
format_stamp: "Formatted at 2026-08-07 22:38:00 on dist-test-slave-7kzw"
server_key: "c523a4e80ff20033d8e644375f540188"
server_key_iv: "854778dfc9c7fb2c08996d9c790b0d20"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:01.693058 14868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:01.707096 14868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:01.709973 14868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:01.710227 14868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:01.710626 14868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:01.711125 14868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:01.711236 14868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.711367 14868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:01.711459 14868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.720566 14868 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:38509
I20260807 22:38:01.721045 14868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:38:01.721771 15084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:38509 every 8 connection(s)
I20260807 22:38:01.727439 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 14868
I20260807 22:38:01.727773 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:44181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:41215
--local_ip_for_outbound_sockets=127.28.82.194
--tserver_master_addrs=127.28.82.254:36881
--webserver_port=37163
--webserver_interface=127.28.82.194
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.82.212:44181/kms with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:01.736575 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:01.721958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36881 (local address 127.28.82.193:56355)
0807 22:38:01.722159 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:01.722207 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:01.722929 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:01.723682 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:01.723688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:01.723924 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:01.724128 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:01.724132 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.725855 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.725857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:01.727313 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:01.727317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.728806 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.728808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:01.728954 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:01.729615 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:01.729628 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:01.731454 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:01.734396 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:01.734403 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:01.734405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:01.734781 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:01.735133 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:01.735134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:01.735134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:01.735285 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:01.735832 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:01.735839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:01.736014 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 22:38:01.736126 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:38:01.736313 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":126,"threads_started":1}
I20260807 22:38:01.737632 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:01.722431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:56355 (local address 127.28.82.254:36881)
0807 22:38:01.723174 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:01.723184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:01.723229 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:38:01.723393 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:01.723401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:01.723463 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:01.723624 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:01.724273 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.725709 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.727441 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.728467 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.728594 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:01.731604 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:01.731609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:01.731610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:01.731625 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:01.734299 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:01.734897 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:01.734900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:01.734901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:01.735023 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:01.735379 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:01.735380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:01.735380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:01.735728 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:01.737236 (+ 1508us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:01.737323 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 22:38:01.737460 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":103,"threads_started":1}
I20260807 22:38:01.740048 15085 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36881
I20260807 22:38:01.740258 15085 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:01.740597 15085 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
W20260807 22:38:01.742321 15085 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:36881 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:38:01.744802 15085 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:01.744998 15085 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:01.745514 15085 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:01.745680 15085 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
W20260807 22:38:01.746168 15085 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:38:01.851871 15089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:01.852105 15089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:01.852150 15089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:01.853581 15089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:01.853648 15089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:01.853706 15089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:38:01.855612 15089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.82.212:44181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:41215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=37163
--tserver_master_addrs=127.28.82.254:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:01.855918 15089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:01.856217 15089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:01.862991 15103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:01.863029 15101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:01.863170 15089 server_base.cc:1034] running on GCE node
W20260807 22:38:01.863171 15097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:01.866828 15089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:01.867161 15089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:01.868510 15089 hybrid_clock.cc:648] HybridClock initialized: now 1786142281868492 us; error 46 us; skew 500 ppm
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:01.872009 15089 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:38:01.872781 15089 webserver.cc:460] Webserver started at http://127.28.82.194:37163/ using document root <none> and password file <none>
I20260807 22:38:01.873091 15089 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:01.873183 15089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:38:01.895120 15089 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 22:38:01.896364 15118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.899696 15089 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260807 22:38:01.899863 15089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "41657cc2387545e1b18556ecfe392c00"
format_stamp: "Formatted at 2026-08-07 22:38:00 on dist-test-slave-7kzw"
server_key: "8db8c194edf1cf364393526c478bfefb"
server_key_iv: "1035f39e6cd9b9e1cc46f6f8990c0450"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:01.900373 15089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:01.917413 15089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:01.919947 15089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:01.920116 15089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:01.920403 15089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:01.920795 15089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:01.920847 15089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.920879 15089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:01.920903 15089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:01.928380 15089 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:41215
I20260807 22:38:01.928841 15089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:38:01.929345 15294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:41215 every 8 connection(s)
I20260807 22:38:01.935837 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 15089
I20260807 22:38:01.936174 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.82.212:44181/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:35799
--local_ip_for_outbound_sockets=127.28.82.195
--tserver_master_addrs=127.28.82.254:36881
--webserver_port=45299
--webserver_interface=127.28.82.195
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.82.212:44181/kms with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:38:01 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142281, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:01.942579 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:01.930128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36881 (local address 127.28.82.194:49289)
0807 22:38:01.930336 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:01.930391 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:01.931090 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:01.931454 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:01.931458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:01.931705 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:01.931979 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:01.931985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.933022 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.933023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:01.934327 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:01.934330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.934945 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.934946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:01.935092 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:01.935722 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:01.935732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:01.938643 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:01.940684 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:01.940692 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:01.940695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:01.941058 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:01.941313 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:01.941313 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:01.941314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:01.941465 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:01.941798 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:01.941804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:01.941975 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 22:38:01.942099 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 22:38:01.942298 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
I20260807 22:38:01.942833 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:01.930130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:49289 (local address 127.28.82.254:36881)
0807 22:38:01.930179 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:01.930181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:01.930542 (+ 361us) server_negotiation.cc:407] Connection header received
0807 22:38:01.931229 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:01.931231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:01.931248 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:01.931317 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:01.932095 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.932903 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.934435 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:01.934774 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:01.934824 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:01.938748 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:01.938750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:01.938750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:01.938760 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:01.940578 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:01.941153 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:01.941155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:01.941155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:01.941222 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:01.941548 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:01.941549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:01.941549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:01.941696 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:01.941765 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:01.942583 (+ 818us) server_negotiation.cc:299] Negotiation successful
0807 22:38:01.942695 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:38:01.943349 15295 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36881
I20260807 22:38:01.943475 15295 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:01.943723 15295 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
W20260807 22:38:01.944478 15295 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:36881 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:38:01.944563 15295 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:01.944655 15295 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:01.945016 15295 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:01.945104 15295 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
W20260807 22:38:01.945498 15295 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:38:02.048785 15298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:02.049171 15298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:02.049257 15298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:02.051069 15298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:02.051186 15298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:02.051278 15298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:38:02.053040 15298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.82.212:44181/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:35799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=45299
--tserver_master_addrs=127.28.82.254:36881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:02.053352 15298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:02.053634 15298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:02.059290 15306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:02.059466 15307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:02.059717 15312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:02.060719 15298 server_base.cc:1034] running on GCE node
I20260807 22:38:02.061069 15298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:02.061362 15298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:02.062538 15298 hybrid_clock.cc:648] HybridClock initialized: now 1786142282062504 us; error 49 us; skew 500 ppm
Aug 07 22:38:02 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142282, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:02.067646 15298 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:38:02.068251 15298 webserver.cc:460] Webserver started at http://127.28.82.195:45299/ using document root <none> and password file <none>
I20260807 22:38:02.068547 15298 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:02.068645 15298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 22:38:02 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142282, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:38:02.093434 15298 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 22:38:02.094282 15330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:02.094521 15298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:02.094609 15298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "38f10e4fb3d54405b27192988ae51979"
format_stamp: "Formatted at 2026-08-07 22:38:01 on dist-test-slave-7kzw"
server_key: "5768ad2c0bb1dc0c8bf5eff53434c350"
server_key_iv: "6ee5f3acbab3be6e6adec5fa17508fd4"
server_key_version: "kuduclusterkey@0"
I20260807 22:38:02.095824 15298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:02.125645 15298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:02.127964 15298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:02.128139 15298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:02.128429 15298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:02.128837 15298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:02.128896 15298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:02.128937 15298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:02.128966 15298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:02.136488 15298 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:35799
I20260807 22:38:02.136593 15496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:35799 every 8 connection(s)
I20260807 22:38:02.136945 15298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:38:02 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142282, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:02.144762 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 15298
I20260807 22:38:02.148443 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:02.137789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:33309 (local address 127.28.82.254:36881)
0807 22:38:02.137873 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:02.137877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:02.138148 (+ 271us) server_negotiation.cc:407] Connection header received
0807 22:38:02.138933 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:02.138934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:02.138944 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:02.138995 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:02.139763 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:02.140250 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:02.141846 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:02.142039 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:02.142097 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:02.144696 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:02.144698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:02.144698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:02.144706 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:02.146662 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:02.147163 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:02.147164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:02.147164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:02.147246 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:02.147554 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:02.147555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:02.147556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:02.147695 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:02.147767 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:02.148226 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 22:38:02.148333 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:38:02.149051 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:02.137751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:36881 (local address 127.28.82.195:33309)
0807 22:38:02.137982 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:02.138041 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:02.138810 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:02.139086 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:02.139091 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:02.139369 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:02.139668 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:02.139674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:02.140336 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:02.140337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:02.141738 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:02.141741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:02.142148 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:02.142149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:02.142298 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:02.142856 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:02.142867 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:02.144573 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:02.146755 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:02.146762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:02.146765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:02.147075 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:02.147332 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:02.147333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:02.147333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:02.147474 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:02.147798 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:02.147805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:02.148021 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 22:38:02.148557 (+ 536us) client_negotiation.cc:240] Negotiation successful
0807 22:38:02.148741 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260807 22:38:02.150027 15497 heartbeater.cc:347] Connected to a master server at 127.28.82.254:36881
I20260807 22:38:02.150141 15497 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:02.150354 15497 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
W20260807 22:38:02.151328 15497 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:36881 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 22:38:02.151405 15497 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:02.151477 15497 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:02.151875 15497 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:02.151954 15497 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
W20260807 22:38:02.152357 15497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:38:02.153235 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:02.145435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33406 (local address 127.28.82.254:36881)
0807 22:38:02.145568 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:02.145570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:02.145725 (+ 155us) server_negotiation.cc:407] Connection header received
0807 22:38:02.145793 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:02.145794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:02.145809 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:02.145887 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:02.146497 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:02.146993 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:02.148018 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:02.148227 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:02.148304 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:02.149843 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:02.149844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:02.149844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:02.149850 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:02.151873 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:02.152406 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:02.152407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:02.152408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:02.152470 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:02.152696 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:02.152697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:02.152698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:02.152810 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:02.152994 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:02.153032 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 22:38:02.153114 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
Aug 07 22:38:02 dist-test-slave-7kzw krb5kdc[13392](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786142282, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:02.746447 15085 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:02.746639 15085 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:02.747085 14810 ts_manager.cc:194] Registered new tserver with Master: a26784e1ec604981a80330b93e625c2b (127.28.82.193:38509)
I20260807 22:38:02.748147 14810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:56355
I20260807 22:38:02.945703 15295 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:02.945876 15295 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:02.946305 14810 ts_manager.cc:194] Registered new tserver with Master: 41657cc2387545e1b18556ecfe392c00 (127.28.82.194:41215)
I20260807 22:38:02.946882 14810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:49289
Aug 07 22:38:03 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142282, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
I20260807 22:38:03.152561 15497 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:03.152733 15497 heartbeater.cc:511] Master 127.28.82.254:36881 requested a full tablet report, sending...
I20260807 22:38:03.153052 14810 ts_manager.cc:194] Registered new tserver with Master: 38f10e4fb3d54405b27192988ae51979 (127.28.82.195:35799)
I20260807 22:38:03.153582 14810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:33309
I20260807 22:38:03.157471 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:38:03.169402 15532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.161073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.28.82.254:36881)
0807 22:38:03.161194 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:03.161197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:03.161367 (+ 170us) server_negotiation.cc:407] Connection header received
0807 22:38:03.161421 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:03.161422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:03.161434 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:03.161511 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:03.162328 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.163138 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.164181 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.164495 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.164564 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.166148 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:03.166149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:03.166150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:03.166158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:03.168088 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:03.168509 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:03.168510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:03.168511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:03.168579 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:03.168833 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:03.168834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:03.168834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:03.168968 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:03.169069 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:03.169210 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 22:38:03.169298 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260807 22:38:03.171825 14810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:38:03.173967 14810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:38:03.187129 15543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.177288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:41215 (local address 127.0.0.1:59398)
0807 22:38:03.177674 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:03.177727 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:03.177804 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:03.183109 (+ 5305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:03.183113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:03.183115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:03.183200 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.183203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.184498 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.184499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.185819 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.185822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.186831 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.186832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.186918 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.186934 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 22:38:03.186982 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 22:38:03.187031 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":99,"threads_started":1}
I20260807 22:38:03.187629 15541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.177673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:38509 (local address 127.0.0.1:46104)
0807 22:38:03.182614 (+ 4941us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:03.182627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:03.182691 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:03.183531 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:03.183532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:03.183534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:03.183581 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.183582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.184956 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.184957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.186366 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.186369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.187407 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.187408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.187471 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.187476 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:38:03.187506 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 22:38:03.187546 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4919,"thread_start_us":61,"threads_started":1}
I20260807 22:38:03.188000 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.177954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.28.82.193:38509)
0807 22:38:03.183344 (+ 5390us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:03.183349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:03.183359 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:38:03.183395 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:03.183397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:03.183415 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:03.183668 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:03.183779 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.184845 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.186504 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.187288 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.187771 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.187842 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:03.187872 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 22:38:03.187917 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5347,"thread_start_us":74,"threads_started":1}
I20260807 22:38:03.188151 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.177405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59398 (local address 127.28.82.194:41215)
0807 22:38:03.182839 (+ 5434us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:03.182843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:03.182854 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:38:03.182895 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:03.182897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:03.182915 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:03.182995 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:03.183282 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.184395 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.185926 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.186729 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.187617 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.187932 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:03.187960 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 22:38:03.188001 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5393,"thread_start_us":66,"threads_started":1}
I20260807 22:38:03.188853 14965 tablet_service.cc:1467] Processing CreateTablet for tablet 511a8fc8ba064a9a947cc0a9af49250b (DEFAULT_TABLE table=test-table [id=279f0e5068474b11a516750eb14850a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:03.188990 15174 tablet_service.cc:1467] Processing CreateTablet for tablet 511a8fc8ba064a9a947cc0a9af49250b (DEFAULT_TABLE table=test-table [id=279f0e5068474b11a516750eb14850a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:03.189262 14965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 511a8fc8ba064a9a947cc0a9af49250b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:03.189352 15174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 511a8fc8ba064a9a947cc0a9af49250b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:03.192196 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.177952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.28.82.195:35799)
0807 22:38:03.182648 (+ 4696us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:03.182655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:03.183014 (+ 359us) server_negotiation.cc:407] Connection header received
0807 22:38:03.183065 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:03.183068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:03.183086 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:03.183172 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:03.188742 (+ 5570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.189721 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.191004 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.191751 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.191804 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.191857 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:03.192075 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 22:38:03.192114 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4648,"thread_start_us":70,"threads_started":1}
I20260807 22:38:03.192440 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.177519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:35799 (local address 127.0.0.1:38436)
0807 22:38:03.182595 (+ 5076us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:03.182611 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:03.182682 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:03.188582 (+ 5900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:03.188583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:03.188586 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:03.188625 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.188627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.189827 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.189828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.190896 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.190899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.191920 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.191921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.191985 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.191988 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:38:03.192310 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 22:38:03.192354 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5047,"thread_start_us":68,"threads_started":1}
I20260807 22:38:03.193100 15396 tablet_service.cc:1467] Processing CreateTablet for tablet 511a8fc8ba064a9a947cc0a9af49250b (DEFAULT_TABLE table=test-table [id=279f0e5068474b11a516750eb14850a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:03.193439 15396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 511a8fc8ba064a9a947cc0a9af49250b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:03.196614 15552 tablet_bootstrap.cc:492] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Bootstrap starting.
I20260807 22:38:03.197386 15552 tablet_bootstrap.cc:654] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:03.197729 15552 log.cc:824] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:03.198101 15548 tablet_bootstrap.cc:492] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00: Bootstrap starting.
I20260807 22:38:03.198619 15549 tablet_bootstrap.cc:492] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b: Bootstrap starting.
I20260807 22:38:03.198768 15548 tablet_bootstrap.cc:654] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:03.199178 15548 log.cc:824] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:03.199283 15549 tablet_bootstrap.cc:654] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:03.199641 15549 log.cc:824] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:03.200040 15548 tablet_bootstrap.cc:492] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00: No bootstrap required, opened a new log
I20260807 22:38:03.200114 15548 ts_tablet_manager.cc:1397] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:38:03.202946 15552 tablet_bootstrap.cc:492] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: No bootstrap required, opened a new log
I20260807 22:38:03.203032 15552 ts_tablet_manager.cc:1397] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 22:38:03.205694 15552 raft_consensus.cc:348] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.205891 15552 raft_consensus.cc:374] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:03.205942 15552 raft_consensus.cc:729] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38f10e4fb3d54405b27192988ae51979, State: Initialized, Role: FOLLOWER
I20260807 22:38:03.206065 15552 consensus_queue.cc:260] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.206353 15552 ts_tablet_manager.cc:1428] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 22:38:03.206602 15497 heartbeater.cc:503] Master 127.28.82.254:36881 was elected leader, sending a full tablet report...
I20260807 22:38:03.206856 15548 raft_consensus.cc:348] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.207003 15548 raft_consensus.cc:374] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:03.207032 15548 raft_consensus.cc:729] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41657cc2387545e1b18556ecfe392c00, State: Initialized, Role: FOLLOWER
I20260807 22:38:03.207258 15548 consensus_queue.cc:260] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.207585 15548 ts_tablet_manager.cc:1428] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260807 22:38:03.207698 15295 heartbeater.cc:503] Master 127.28.82.254:36881 was elected leader, sending a full tablet report...
I20260807 22:38:03.208635 15549 tablet_bootstrap.cc:492] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b: No bootstrap required, opened a new log
I20260807 22:38:03.208727 15549 ts_tablet_manager.cc:1397] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260807 22:38:03.211318 15549 raft_consensus.cc:348] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.211481 15549 raft_consensus.cc:374] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:03.211535 15549 raft_consensus.cc:729] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26784e1ec604981a80330b93e625c2b, State: Initialized, Role: FOLLOWER
I20260807 22:38:03.211673 15549 consensus_queue.cc:260] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.211958 15549 ts_tablet_manager.cc:1428] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:38:03.212110 15085 heartbeater.cc:503] Master 127.28.82.254:36881 was elected leader, sending a full tablet report...
W20260807 22:38:03.222621 15086 tablet.cc:2369] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 22:38:03 dist-test-slave-7kzw krb5kdc[13392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142282, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for HTTP/127.28.82.212@KRBTEST.COM
W20260807 22:38:03.388384 15498 tablet.cc:2369] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:38:03.418515 15567 raft_consensus.cc:482] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:03.418679 15567 raft_consensus.cc:504] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.419099 15567 leader_election.cc:288] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41657cc2387545e1b18556ecfe392c00 (127.28.82.194:41215), 38f10e4fb3d54405b27192988ae51979 (127.28.82.195:35799)
W20260807 22:38:03.430577 15296 tablet.cc:2369] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:38:03.431860 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.419953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:35117 (local address 127.28.82.194:41215)
0807 22:38:03.420021 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:03.420024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:03.421907 (+ 1883us) server_negotiation.cc:407] Connection header received
0807 22:38:03.422018 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:03.422019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:03.422030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:03.422060 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:03.422354 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.423110 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.430845 (+ 7735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.431440 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.431491 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.431500 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:03.431735 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 22:38:03.431785 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 22:38:03.432314 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.419414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:41215 (local address 127.28.82.193:35117)
0807 22:38:03.421830 (+ 2416us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:03.421844 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:03.421939 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:03.422131 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:03.422132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:03.422134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:03.422229 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.422232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.424532 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.424533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.425864 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.425867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.431573 (+ 5706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.431574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.431646 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.431650 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:38:03.432155 (+ 505us) client_negotiation.cc:240] Negotiation successful
0807 22:38:03.432198 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2394,"thread_start_us":102,"threads_started":1}
I20260807 22:38:03.432798 15202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "511a8fc8ba064a9a947cc0a9af49250b" candidate_uuid: "a26784e1ec604981a80330b93e625c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41657cc2387545e1b18556ecfe392c00" is_pre_election: true
I20260807 22:38:03.432981 15202 raft_consensus.cc:2454] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a26784e1ec604981a80330b93e625c2b in term 0.
I20260807 22:38:03.433239 14931 leader_election.cc:302] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41657cc2387545e1b18556ecfe392c00, a26784e1ec604981a80330b93e625c2b; no voters:
I20260807 22:38:03.433406 15567 raft_consensus.cc:2781] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:38:03.433472 15567 raft_consensus.cc:482] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:38:03.433498 15567 raft_consensus.cc:3037] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:03.434481 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.420035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:55393 (local address 127.28.82.195:35799)
0807 22:38:03.420088 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:03.420090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:03.422608 (+ 2518us) server_negotiation.cc:407] Connection header received
0807 22:38:03.422716 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:03.422718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:03.422727 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:03.422757 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:03.430926 (+ 8169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.431602 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.433462 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.434018 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.434072 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.434081 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:03.434318 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 22:38:03.434372 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 22:38:03.434499 15567 raft_consensus.cc:504] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.434633 15567 leader_election.cc:288] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [CANDIDATE]: Term 1 election: Requested vote from peers 41657cc2387545e1b18556ecfe392c00 (127.28.82.194:41215), 38f10e4fb3d54405b27192988ae51979 (127.28.82.195:35799)
I20260807 22:38:03.434724 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.419622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:35799 (local address 127.28.82.193:55393)
0807 22:38:03.422533 (+ 2911us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:03.422545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:03.422635 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:03.422819 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:03.422819 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:03.422821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:03.422881 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.422882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.431698 (+ 8816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.431699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.433351 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:03.433354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.434147 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.434148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:03.434229 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.434232 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:38:03.434560 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 22:38:03.434600 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2893,"thread_start_us":78,"threads_started":1}
I20260807 22:38:03.435307 15202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "511a8fc8ba064a9a947cc0a9af49250b" candidate_uuid: "a26784e1ec604981a80330b93e625c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41657cc2387545e1b18556ecfe392c00"
I20260807 22:38:03.435386 15202 raft_consensus.cc:3037] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:03.436285 15202 raft_consensus.cc:2454] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a26784e1ec604981a80330b93e625c2b in term 1.
I20260807 22:38:03.436555 15422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "511a8fc8ba064a9a947cc0a9af49250b" candidate_uuid: "a26784e1ec604981a80330b93e625c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38f10e4fb3d54405b27192988ae51979" is_pre_election: true
I20260807 22:38:03.436657 14931 leader_election.cc:302] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41657cc2387545e1b18556ecfe392c00, a26784e1ec604981a80330b93e625c2b; no voters:
I20260807 22:38:03.436715 15422 raft_consensus.cc:2454] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a26784e1ec604981a80330b93e625c2b in term 0.
I20260807 22:38:03.436805 15567 raft_consensus.cc:2781] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:38:03.436832 15421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "511a8fc8ba064a9a947cc0a9af49250b" candidate_uuid: "a26784e1ec604981a80330b93e625c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38f10e4fb3d54405b27192988ae51979"
I20260807 22:38:03.436885 15421 raft_consensus.cc:3037] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:03.436964 15567 raft_consensus.cc:686] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [term 1 LEADER]: Becoming Leader. State: Replica: a26784e1ec604981a80330b93e625c2b, State: Running, Role: LEADER
I20260807 22:38:03.437055 15567 consensus_queue.cc:237] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } }
I20260807 22:38:03.437740 15421 raft_consensus.cc:2454] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a26784e1ec604981a80330b93e625c2b in term 1.
I20260807 22:38:03.437908 14810 catalog_manager.cc:5495] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b reported cstate change: term changed from 0 to 1, leader changed from <none> to a26784e1ec604981a80330b93e625c2b (127.28.82.193). New cstate: current_term: 1 leader_uuid: "a26784e1ec604981a80330b93e625c2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a26784e1ec604981a80330b93e625c2b" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 38509 } health_report { overall_health: HEALTHY } } }
I20260807 22:38:03.483314 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.479834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.28.82.193:38509)
0807 22:38:03.479909 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:03.479911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:03.479919 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:38:03.479969 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:03.479970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:03.479990 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:03.480081 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:38:03.480243 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.481016 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.482118 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:03.482436 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:03.482566 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:03.482756 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:38:03.482898 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:38:03.482970 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:03.483193 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 22:38:03.483224 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 22:38:03.491873 15421 raft_consensus.cc:1261] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 1 FOLLOWER]: Refusing update from remote peer a26784e1ec604981a80330b93e625c2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:03.492400 15575 consensus_queue.cc:1035] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "38f10e4fb3d54405b27192988ae51979" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:03.495008 15202 raft_consensus.cc:1261] T 511a8fc8ba064a9a947cc0a9af49250b P 41657cc2387545e1b18556ecfe392c00 [term 1 FOLLOWER]: Refusing update from remote peer a26784e1ec604981a80330b93e625c2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:03.495584 15575 consensus_queue.cc:1035] T 511a8fc8ba064a9a947cc0a9af49250b P a26784e1ec604981a80330b93e625c2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "41657cc2387545e1b18556ecfe392c00" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:03.499554 15578 mvcc.cc:205] Tried to move back new op lower bound from 7316038793177223168 to 7316038792957095936. Current Snapshot: MvccSnapshot[applied={T|T < 7316038793177223168}]
I20260807 22:38:03.500192 15581 mvcc.cc:205] Tried to move back new op lower bound from 7316038793177223168 to 7316038792957095936. Current Snapshot: MvccSnapshot[applied={T|T < 7316038793177223168}]
I20260807 22:38:03.503780 14810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:03.503901 14810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:03.504695 14810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 937a24ad67d748d09ead70092c9ac4a8: Sending DeleteTablet for 3 replicas of tablet 511a8fc8ba064a9a947cc0a9af49250b
I20260807 22:38:03.505185 15396 tablet_service.cc:1514] Processing DeleteTablet for tablet 511a8fc8ba064a9a947cc0a9af49250b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:03 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:38436
I20260807 22:38:03.505440 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 14868
I20260807 22:38:03.505438 14965 tablet_service.cc:1514] Processing DeleteTablet for tablet 511a8fc8ba064a9a947cc0a9af49250b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:03 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:46104
I20260807 22:38:03.510916 15174 tablet_service.cc:1514] Processing DeleteTablet for tablet 511a8fc8ba064a9a947cc0a9af49250b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:03 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:59398
W20260807 22:38:03.515730 14786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:38509 (error 108)
I20260807 22:38:03.515964 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 15089
I20260807 22:38:03.526597 15590 tablet_replica.cc:330] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: stopping tablet replica
I20260807 22:38:03.526762 15590 raft_consensus.cc:2229] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:03.526901 15590 raft_consensus.cc:2258] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:03.527025 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.526877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:38509 (local address 127.0.0.1:42028)
0807 22:38:03.526934 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:03.526980 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:38509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 22:38:03.527153 14786 catalog_manager.cc:4584] TS a26784e1ec604981a80330b93e625c2b (127.28.82.193:38509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 511a8fc8ba064a9a947cc0a9af49250b: Network error: Client connection negotiation failed: client connection to 127.28.82.193:38509: connect: Connection refused (error 111)
I20260807 22:38:03.527521 15590 ts_tablet_manager.cc:1905] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:03.529318 15590 ts_tablet_manager.cc:1918] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:03.529404 15590 log.cc:1196] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/511a8fc8ba064a9a947cc0a9af49250b
I20260807 22:38:03.529703 15590 ts_tablet_manager.cc:1939] T 511a8fc8ba064a9a947cc0a9af49250b P 38f10e4fb3d54405b27192988ae51979: Deleting consensus metadata
I20260807 22:38:03.530220 14786 catalog_manager.cc:4841] TS 38f10e4fb3d54405b27192988ae51979 (127.28.82.195:35799): tablet 511a8fc8ba064a9a947cc0a9af49250b (table test-table [id=279f0e5068474b11a516750eb14850a5]) successfully deleted
I20260807 22:38:03.532868 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 15298
I20260807 22:38:03.542074 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:03.540635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:41215 (local address 127.0.0.1:45480)
0807 22:38:03.541999 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:03.542039 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.194:41215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1337}
I20260807 22:38:03.547735 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 14761
I20260807 22:38:03.557356 29003 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 22:38:03.901935 29003 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 22:38:03.902031 29003 mini_ranger.cc:66] Stopping Ranger...
I20260807 22:38:04.246604 29003 mini_ranger.cc:68] Stopped Ranger
2026-08-07 22:38:04.246 UTC [13435] LOG: received smart shutdown request
2026-08-07 22:38:04.251 UTC [13435] LOG: background worker "logical replication launcher" (PID 13447) exited with exit code 1
2026-08-07 22:38:04.251 UTC [13440] LOG: shutting down
2026-08-07 22:38:04.419 UTC [13435] LOG: database system is shut down
2026-08-07T22:38:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51767 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:38:04 dist-test-slave-7kzw krb5kdc[15609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:38:04 dist-test-slave-7kzw krb5kdc[15609](info): set up 2 sockets
Aug 07 22:38:04 dist-test-slave-7kzw krb5kdc[15609](info): commencing operation
krb5kdc: starting...
W20260807 22:38:05.802762 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:38:05 dist-test-slave-7kzw krb5kdc[15609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:38:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:38:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:38:05.927561 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39851
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39851,127.28.82.253:35257,127.28.82.252:41157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:06.011409 15628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:06.011626 15628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:06.011649 15628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:06.012814 15628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:06.012861 15628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:06.012874 15628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:06.012888 15628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:06.012900 15628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:06.014571 15628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39851,127.28.82.253:35257,127.28.82.252:41157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:06.014835 15628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:06.015094 15628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:06.020028 15638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:06.020069 15634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:06.020028 15635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:06.020431 15628 server_base.cc:1034] running on GCE node
I20260807 22:38:06.020780 15628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:06.021128 15628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:06.022302 15628 hybrid_clock.cc:648] HybridClock initialized: now 1786142286022269 us; error 43 us; skew 500 ppm
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:06.025089 15628 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:38:06.025520 15628 webserver.cc:460] Webserver started at http://127.28.82.254:34617/ using document root <none> and password file <none>
I20260807 22:38:06.025806 15628 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:06.025882 15628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:06.026032 15628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:06.027182 15628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "8a315fe9f3da404a9d17e2db4368d24e"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "27331de8af5a6be7abef5b094c1bb54e"
server_key_iv: "7efa5d9f8e59e5fa481ae373d92015d8"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.027572 15628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "8a315fe9f3da404a9d17e2db4368d24e"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "27331de8af5a6be7abef5b094c1bb54e"
server_key_iv: "7efa5d9f8e59e5fa481ae373d92015d8"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.029018 15628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:38:06.029917 15646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:06.030268 15628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:06.030382 15628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "8a315fe9f3da404a9d17e2db4368d24e"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "27331de8af5a6be7abef5b094c1bb54e"
server_key_iv: "7efa5d9f8e59e5fa481ae373d92015d8"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.030459 15628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:06.048977 15628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:06.051532 15628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:06.051702 15628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:06.056005 15628 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:39851
I20260807 22:38:06.056046 15727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:39851 every 8 connection(s)
I20260807 22:38:06.056420 15628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:38:06.057339 15728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:06.059034 15728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:38:06.062891 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 15628
I20260807 22:38:06.063020 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:38:06.063211 29003 external_mini_cluster.cc:1428] Setting key 0d1937c2857041cd81c5712366319f64
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:06.069936 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.059718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39851 (local address 127.0.0.1:52828)
0807 22:38:06.059960 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.059987 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.060424 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.060712 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.060714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.060960 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.061110 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.061114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.062108 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.062108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.062857 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.062859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.063505 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.063506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.063576 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.064183 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.064193 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.065945 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.068275 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.068286 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.068288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.068578 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.068882 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.068883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.068883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.068978 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.069460 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.069466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.069535 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.069633 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.069769 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"spinlock_wait_cycles":8448,"thread_start_us":92,"threads_started":1}
I20260807 22:38:06.070065 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.059736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.28.82.254:39851)
0807 22:38:06.059898 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.059907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.060066 (+ 159us) server_negotiation.cc:407] Connection header received
0807 22:38:06.060549 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.060551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.060568 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.060650 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.061197 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.062022 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.062959 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.063409 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.063531 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.066035 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.066043 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.066043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.066050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.068144 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.068677 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.068681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.068682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.068805 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.069055 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.069056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.069056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.069372 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.069473 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.069699 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.069867 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260807 22:38:06.071202 15728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:38:06.071591 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.071459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54474)
0807 22:38:06.071508 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.071559 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:38:06.071775 15653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
I20260807 22:38:06.072274 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.072176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54486)
0807 22:38:06.072228 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.072256 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 22:38:06.072418 15728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:35257: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.28.82.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab.
Entry for principal kudu/127.28.82.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab.
I20260807 22:38:06.089306 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.253:35257
--webserver_interface=127.28.82.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab
--principal=kudu/127.28.82.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39851,127.28.82.253:35257,127.28.82.252:41157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:06.121691 15728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } attempt: 1
I20260807 22:38:06.122184 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.122033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54490)
0807 22:38:06.122103 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.122153 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 22:38:06.122555 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.122442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54492)
0807 22:38:06.122499 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.122528 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 22:38:06.122653 15728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:35257: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
W20260807 22:38:06.172283 15741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:06.172523 15741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:06.172559 15741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:06.173686 15741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:06.173759 15741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:06.173785 15741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:06.173805 15741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:06.173837 15741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:06.175426 15741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39851,127.28.82.253:35257,127.28.82.252:41157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab
--principal=kudu/127.28.82.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.253:35257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.82.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:06.175700 15741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:06.175933 15741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:06.180904 15753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:06.180904 15755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:06.180954 15752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:06.181159 15741 server_base.cc:1034] running on GCE node
I20260807 22:38:06.181450 15741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:06.181747 15741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:06.182915 15741 hybrid_clock.cc:648] HybridClock initialized: now 1786142286182901 us; error 27 us; skew 500 ppm
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:06.185523 15741 init.cc:377] Logged in from keytab as kudu/127.28.82.253@KRBTEST.COM (short username kudu)
I20260807 22:38:06.185905 15741 webserver.cc:460] Webserver started at http://127.28.82.253:43473/ using document root <none> and password file <none>
I20260807 22:38:06.186075 15741 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:06.186115 15741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:06.186200 15741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:06.187211 15741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/data/instance:
uuid: "3963e1c6e0d94437ac43dd20b5bd1a69"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "3e6daa1673ea006100c34ba776336275"
server_key_iv: "0ac1663030072bed21e4addf95299f7c"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.187544 15741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/wal/instance:
uuid: "3963e1c6e0d94437ac43dd20b5bd1a69"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "3e6daa1673ea006100c34ba776336275"
server_key_iv: "0ac1663030072bed21e4addf95299f7c"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.188796 15741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:38:06.189488 15762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:06.189689 15741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:38:06.189792 15741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/wal
uuid: "3963e1c6e0d94437ac43dd20b5bd1a69"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "3e6daa1673ea006100c34ba776336275"
server_key_iv: "0ac1663030072bed21e4addf95299f7c"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.189872 15741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:06.190791 15728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } attempt: 2
I20260807 22:38:06.191171 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.191037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54498)
0807 22:38:06.191101 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.191139 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 22:38:06.191515 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.191431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54512)
0807 22:38:06.191474 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.191499 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 22:38:06.191625 15728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:35257: Network error: Client connection negotiation failed: client connection to 127.28.82.253:35257: connect: Connection refused (error 111)
I20260807 22:38:06.237488 15741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:06.239955 15741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:06.240123 15741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:06.244693 15741 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.253:35257
I20260807 22:38:06.244699 15839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.253:35257 every 8 connection(s)
I20260807 22:38:06.245090 15741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
I20260807 22:38:06.245968 15841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:06.247632 15841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:06.254817 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 15741
I20260807 22:38:06.254937 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-1/wal/instance
I20260807 22:38:06.255174 29003 external_mini_cluster.cc:1428] Setting key 1447803c59c02a4b2ae9618d5c19485f
I20260807 22:38:06.256960 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.248398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52842 (local address 127.28.82.254:39851)
0807 22:38:06.248463 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.248466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.248604 (+ 138us) server_negotiation.cc:407] Connection header received
0807 22:38:06.249086 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.249087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.249100 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.249167 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.249762 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.250273 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.251403 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.251615 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.251683 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.253882 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.253883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.253884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.253891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.255474 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.255933 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.255935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.255936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.256008 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.256255 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.256256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.256257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.256410 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.256483 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.256718 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.256814 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:38:06.256976 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.248308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39851 (local address 127.0.0.1:52842)
0807 22:38:06.248478 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.248519 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.248989 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.249216 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.249219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.249455 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.249651 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.249655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.250383 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.250389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.251302 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.251305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.251714 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.251715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.251823 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.252320 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.252329 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.253786 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.255579 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.255586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.255588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.255847 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.256083 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.256083 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.256084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.256189 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.256477 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.256481 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.256594 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.256673 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.256820 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260807 22:38:06.257813 15841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.82.252@KRBTEST.COM; defaulting to no policy
I20260807 22:38:06.265121 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.258089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54516)
0807 22:38:06.258138 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.258149 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.258188 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.258510 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.258511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.258653 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.258700 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.258702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.259601 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.259603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.260090 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.260091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.260564 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.260565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.260632 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.261047 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.261052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.262201 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.263924 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.263925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.263926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.264129 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.264383 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.264383 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.264384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.264440 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.264738 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.264739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.264784 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.264865 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.264962 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:38:06.265213 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.258198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.28.82.253:35257)
0807 22:38:06.258354 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.258357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.258374 (+ 17us) server_negotiation.cc:407] Connection header received
0807 22:38:06.258390 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.258392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.258402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.258512 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.258814 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.259491 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.260185 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.260486 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.260569 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.262288 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.262296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.262297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.262303 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.263823 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.264234 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.264237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.264237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.264304 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.264510 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.264511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.264511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.264655 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.264746 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.264900 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.265035 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260807 22:38:06.265887 15841 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:38:06.266286 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.266130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46916)
0807 22:38:06.266180 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.266231 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 22:38:06.266500 15764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.252@KRBTEST.COM" created.
I20260807 22:38:06.267063 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.266951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46930)
0807 22:38:06.267001 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.267032 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 22:38:06.267190 15841 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41157: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab.
Entry for principal kudu/127.28.82.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab.
I20260807 22:38:06.282613 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.252:41157
--webserver_interface=127.28.82.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab
--principal=kudu/127.28.82.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:41515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39851,127.28.82.253:35257,127.28.82.252:41157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:06.282769 15728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } attempt: 3
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:38:06.290321 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.283129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54526)
0807 22:38:06.283202 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.283227 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.283275 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.283438 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.283439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.283591 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.283656 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.283657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.284342 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.284343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.284856 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.284858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.285245 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.285246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.285330 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.285781 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.285788 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.287257 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.289273 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.289274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.289275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.289442 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.289695 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.289695 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.289696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.289754 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.290082 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.290083 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.290122 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.290177 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.290253 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 22:38:06.290449 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.283191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.28.82.253:35257)
0807 22:38:06.283254 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.283256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.283267 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:38:06.283340 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.283341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.283351 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.283420 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.283739 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.284242 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.284946 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.285168 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.285249 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.287358 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.287359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.287360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.287369 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.289181 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.289518 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.289519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.289519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.289589 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.289822 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.289823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.289824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.289979 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.290082 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.290224 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.290316 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:38:06.291040 15728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:38:06.291416 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.291311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46942)
0807 22:38:06.291361 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.291393 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 22:38:06.291803 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.291718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46944)
0807 22:38:06.291763 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.291787 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:38:06.291923 15728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41157: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
I20260807 22:38:06.316468 15841 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } attempt: 1
I20260807 22:38:06.316995 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.316855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46954)
0807 22:38:06.316921 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.316961 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 22:38:06.317435 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.317342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46964)
0807 22:38:06.317385 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.317414 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 22:38:06.317548 15841 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41157: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
I20260807 22:38:06.323093 15728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } attempt: 1
I20260807 22:38:06.323521 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.323395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46968)
0807 22:38:06.323451 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.323490 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 22:38:06.323925 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.323831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46982)
0807 22:38:06.323874 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.323903 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 22:38:06.324023 15728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41157: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
W20260807 22:38:06.369370 15851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:06.369575 15851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:06.369597 15851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:06.370784 15851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:06.370827 15851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:06.370842 15851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:06.370855 15851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:06.370868 15851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:06.372411 15851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39851,127.28.82.253:35257,127.28.82.252:41157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab
--principal=kudu/127.28.82.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.252:41157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.82.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:06.372658 15851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:06.372910 15851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:06.377871 15859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:06.377888 15860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:06.377884 15863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:06.378641 15851 server_base.cc:1034] running on GCE node
I20260807 22:38:06.378881 15851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:06.379161 15851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:06.380328 15851 hybrid_clock.cc:648] HybridClock initialized: now 1786142286380312 us; error 52 us; skew 500 ppm
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:06.382889 15851 init.cc:377] Logged in from keytab as kudu/127.28.82.252@KRBTEST.COM (short username kudu)
I20260807 22:38:06.383277 15851 webserver.cc:460] Webserver started at http://127.28.82.252:35317/ using document root <none> and password file <none>
I20260807 22:38:06.383452 15851 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:06.383491 15851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:06.383579 15851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:06.384541 15851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/data/instance:
uuid: "f08c9562c918471ca62f4d1a6d172f76"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "9c9a4c0bcd145691beb3ab2ffa031e1f"
server_key_iv: "db9cb9028ff98699e89ee4955f16e1c4"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.384850 15851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/wal/instance:
uuid: "f08c9562c918471ca62f4d1a6d172f76"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "9c9a4c0bcd145691beb3ab2ffa031e1f"
server_key_iv: "db9cb9028ff98699e89ee4955f16e1c4"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.385715 15841 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } attempt: 2
I20260807 22:38:06.386210 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.386029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46994)
0807 22:38:06.386121 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.386170 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 22:38:06.386250 15851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:38:06.386716 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.386620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:46996)
0807 22:38:06.386660 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.386689 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 22:38:06.386860 15841 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41157: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
I20260807 22:38:06.387444 15876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:06.387666 15851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:38:06.387799 15851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/wal
uuid: "f08c9562c918471ca62f4d1a6d172f76"
format_stamp: "Formatted at 2026-08-07 22:38:06 on dist-test-slave-7kzw"
server_key: "9c9a4c0bcd145691beb3ab2ffa031e1f"
server_key_iv: "db9cb9028ff98699e89ee4955f16e1c4"
server_key_version: "encryptionkey@0"
I20260807 22:38:06.387887 15851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:06.399173 15728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } attempt: 2
I20260807 22:38:06.399598 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.399469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:47012)
0807 22:38:06.399529 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.399560 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 22:38:06.399911 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.399813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:47018)
0807 22:38:06.399864 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.399889 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 22:38:06.400005 15728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41157: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41157: connect: Connection refused (error 111)
I20260807 22:38:06.414273 15851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:06.416618 15851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:06.416824 15851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:06.421705 15851 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.252:41157
I20260807 22:38:06.421734 15962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.252:41157 every 8 connection(s)
I20260807 22:38:06.422178 15851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
I20260807 22:38:06.423177 15965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:06.424728 15965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:38:06.428812 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 15851
I20260807 22:38:06.428964 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786142120597695-29003-0/minicluster-data/master-2/wal/instance
I20260807 22:38:06.429167 29003 external_mini_cluster.cc:1428] Setting key b6b06621e73e7cbb94998105d0293435
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:38:06.435379 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.425489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.28.82.254:39851)
0807 22:38:06.425587 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.425590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.425802 (+ 212us) server_negotiation.cc:407] Connection header received
0807 22:38:06.426369 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.426371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.426385 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.426444 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.427191 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.427737 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.428882 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.429215 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.429317 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.431736 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.431738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.431738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.431749 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.433788 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.434233 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.434235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.434235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.434307 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.434653 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.434654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.434655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.434828 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.434894 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.435246 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.435316 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:38:06.435603 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.425405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39851 (local address 127.0.0.1:52856)
0807 22:38:06.425655 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.425706 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.426245 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.426577 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.426581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.426860 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.427073 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.427077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.427868 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.427869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.428752 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.428754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.429331 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.429333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.429465 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.430087 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.430096 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.431631 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.433912 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.433919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.433921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.434162 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.434385 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.434385 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.434385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.434583 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.434911 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.434918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.435076 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.435195 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.435359 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260807 22:38:06.436287 15965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:38:06.439224 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.429677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.28.82.252:41157)
0807 22:38:06.429851 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.429857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.430003 (+ 146us) server_negotiation.cc:407] Connection header received
0807 22:38:06.430069 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.430072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.430101 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.430190 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.430870 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.431709 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.432485 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.432995 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.433100 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.435930 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.435937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.435937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.435944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.438015 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.438396 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.438398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.438398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.438452 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.438689 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.438690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.438690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.438823 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.438989 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.439033 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.439109 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":116,"threads_started":1}
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:38:06.443701 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.436670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54540 (local address 127.28.82.253:35257)
0807 22:38:06.436751 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.436753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.436764 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:38:06.436791 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.436792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.436803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.436872 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.437192 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.437902 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.438872 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.439162 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.439227 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.440910 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.440912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.440912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.440919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.442505 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.442869 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.442870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.442871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.442953 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.443187 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.443188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.443189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.443338 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.443402 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.443567 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.443638 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:38:06.443771 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.436583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:35257 (local address 127.0.0.1:54540)
0807 22:38:06.436641 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.436653 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.436696 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.436879 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.436880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.437054 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.437111 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.437113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.437984 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.437985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.438796 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.438799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.439241 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.439242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.439317 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.439708 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.439712 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.440850 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.442585 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.442586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.442586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.442754 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.443067 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.443067 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.443068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.443121 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.443419 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.443419 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.443462 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.443551 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.443642 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 22:38:06.444144 15965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:38:06.448200 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.440181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.28.82.254:39851)
0807 22:38:06.440224 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.440226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.440235 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:38:06.440255 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.440256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.440264 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.440320 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.440835 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.441498 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.442152 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.442512 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.442591 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.445415 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.445416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.445417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.445424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.447016 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.447386 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.447387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.447387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.447437 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.447641 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.447641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.447642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.447778 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.447978 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.448016 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.448100 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:38:06.451191 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.444384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:47040)
0807 22:38:06.444470 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.444479 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.444523 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.444668 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.444669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.444813 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.444847 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.444849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.445622 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.445623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.446062 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.446064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.446582 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.446583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.446625 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.447003 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.447007 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.448507 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.450234 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.450235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.450235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.450439 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.450681 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.450682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.450682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.450736 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.450978 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.450979 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.451012 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.451064 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.451136 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 22:38:06.451292 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.444439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.28.82.252:41157)
0807 22:38:06.444566 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.444568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.444572 (+ 4us) server_negotiation.cc:407] Connection header received
0807 22:38:06.444584 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.444585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.444593 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.444631 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.444946 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.445537 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.446140 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.446452 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.446579 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.448593 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.448594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.448594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.448600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.450153 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.450553 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.450554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.450555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.450616 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.450792 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.450793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.450793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.450914 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.450978 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.451087 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.451195 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":117}
I20260807 22:38:06.452664 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76: Bootstrap starting.
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:38:06.453552 15965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:06.453851 15965 log.cc:824] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:06.454804 15965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76: No bootstrap required, opened a new log
I20260807 22:38:06.456211 15965 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.456354 15965 raft_consensus.cc:374] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:06.456421 15965 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08c9562c918471ca62f4d1a6d172f76, State: Initialized, Role: FOLLOWER
I20260807 22:38:06.456585 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.449196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.28.82.253:35257)
0807 22:38:06.449248 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.449250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.449262 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:38:06.449281 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.449282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.449290 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.449342 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.449929 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.450504 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.451442 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.451695 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.451739 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.453925 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.453927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.453927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.453934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.455406 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.455761 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.455762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.455762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.455824 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.456023 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.456024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.456024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.456138 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.456356 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.456396 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.456469 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:38:06.456553 15965 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.456918 15965 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:06.456987 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.457087 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:38:06.458268 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 22:38:06.458878 15987 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:38:06.458985 15987 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:38:06.459182 15851 master_runner.cc:186] Error getting master registration for 127.28.82.254:39851: OK, instance_id { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" instance_seqno: 1786142286054797 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:38:06.461719 15851 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:38:06.470283 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.459805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47044 (local address 127.28.82.252:41157)
0807 22:38:06.459843 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.459846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.462070 (+ 2224us) server_negotiation.cc:407] Connection header received
0807 22:38:06.462313 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.462314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.462324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.462353 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.463212 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.463900 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.465106 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.465396 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.465435 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.467032 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.467033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.467033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.467040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.468681 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.469087 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.469088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.469088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.469144 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.469633 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.469634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.469634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.469798 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.469876 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.470101 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.470181 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 22:38:06.470402 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.459800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52864 (local address 127.28.82.254:39851)
0807 22:38:06.459837 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.459840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.461896 (+ 2056us) server_negotiation.cc:407] Connection header received
0807 22:38:06.461967 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.461968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.461979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.462038 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.462767 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.463478 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.464507 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.464797 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.464838 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.466343 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.466345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.466345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.466352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.467971 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.469398 (+ 1427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.469399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.469399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.469466 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.469697 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.469698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.469699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.469819 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.469893 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.470158 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.470277 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 22:38:06.471446 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.460162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.28.82.253:35257)
0807 22:38:06.460194 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.460196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.462179 (+ 1983us) server_negotiation.cc:407] Connection header received
0807 22:38:06.462242 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.462243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.462253 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.462298 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.463212 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.463910 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.465101 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.465645 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.465691 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.468443 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.468444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.468445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.468452 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.470122 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.470535 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.470537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.470537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.470599 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.470796 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.470797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.470797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.470918 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.471016 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.471271 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.471350 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 22:38:06.478030 15841 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } attempt: 3
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:38:06.485375 15845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.478356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:47048)
0807 22:38:06.478419 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.478431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.478510 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.478660 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.478661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.478806 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.478840 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.478841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.479468 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.479469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.480031 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.480033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.480407 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.480408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.480471 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.481142 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.481147 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.482762 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.484310 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.484311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.484312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.484491 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.484701 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.484702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.484702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.484775 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.485034 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.485036 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.485088 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.485174 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.485274 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 22:38:06.485375 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.478436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47048 (local address 127.28.82.252:41157)
0807 22:38:06.478493 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.478496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.478505 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:38:06.478572 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.478573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.478582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.478631 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.478925 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.479380 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.480112 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.480313 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.480405 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.482859 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.482860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.482860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.482868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.484217 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.484567 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.484568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.484568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.484629 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.484848 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.484849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.484850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.484957 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.485034 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.485216 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.485288 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:38:06.486850 15841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69: Bootstrap starting.
I20260807 22:38:06.487522 15841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:06.487852 15841 log.cc:824] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:06.488664 15841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69: No bootstrap required, opened a new log
I20260807 22:38:06.490896 15841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.491034 15841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:06.491058 15841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3963e1c6e0d94437ac43dd20b5bd1a69, State: Initialized, Role: FOLLOWER
I20260807 22:38:06.491207 15841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.491647 15841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:06.491623 16006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.491729 16006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:38:06.493774 16021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:38:06.493887 16021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:38:06.493927 15741 master_runner.cc:186] Error getting master registration for 127.28.82.254:39851: OK, instance_id { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" instance_seqno: 1786142286054797 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:38:06.496263 15741 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:38:06.499157 15728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } attempt: 3
Aug 07 22:38:06 dist-test-slave-7kzw krb5kdc[15609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:38:06.506336 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.499443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41157 (local address 127.0.0.1:47064)
0807 22:38:06.499501 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:06.499511 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:06.499563 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:06.499690 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:06.499690 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:06.499857 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:06.499899 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.499900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.500534 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.500535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.501032 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:06.501034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.501410 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.501411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:06.501475 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.501962 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:06.501967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:06.503261 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:06.505271 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.505272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.505272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.505453 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.505668 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:06.505669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:06.505669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:06.505734 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:06.506006 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:06.506007 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:06.506053 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:38:06.506138 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 22:38:06.506237 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 22:38:06.506335 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:06.499502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.28.82.252:41157)
0807 22:38:06.499566 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:06.499568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:06.499577 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:38:06.499620 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:06.499622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:06.499632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:06.499688 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:06.499977 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.500447 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.501125 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:06.501321 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:06.501410 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:06.503351 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:06.503353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:06.503353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:06.503361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:06.505191 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.505529 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.505531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.505531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.505595 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:06.505810 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:06.505811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:06.505811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:06.505928 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:06.506001 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:06.506172 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 22:38:06.506241 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:38:06.507730 15728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: Bootstrap starting.
I20260807 22:38:06.508406 15728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:06.508729 15728 log.cc:824] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:06.509538 15728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: No bootstrap required, opened a new log
I20260807 22:38:06.511096 15728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.511238 15728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:06.511322 15728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a315fe9f3da404a9d17e2db4368d24e, State: Initialized, Role: FOLLOWER
I20260807 22:38:06.511483 15728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.511759 16026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.511821 15728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:06.511898 16026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:38:06.513734 16041 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:38:06.513842 16041 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:38:06.516338 15628 master_runner.cc:263] No leader master found from master 8a315fe9f3da404a9d17e2db4368d24e
I20260807 22:38:06.516387 15628 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:38:06.540210 15975 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:06.540333 15975 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.540555 15975 leader_election.cc:288] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a315fe9f3da404a9d17e2db4368d24e (127.28.82.254:39851), 3963e1c6e0d94437ac43dd20b5bd1a69 (127.28.82.253:35257)
I20260807 22:38:06.541054 15691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08c9562c918471ca62f4d1a6d172f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a315fe9f3da404a9d17e2db4368d24e" is_pre_election: true
I20260807 22:38:06.541193 15691 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f08c9562c918471ca62f4d1a6d172f76 in term 0.
I20260807 22:38:06.541489 15881 leader_election.cc:302] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a315fe9f3da404a9d17e2db4368d24e, f08c9562c918471ca62f4d1a6d172f76; no voters:
I20260807 22:38:06.541829 15797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08c9562c918471ca62f4d1a6d172f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" is_pre_election: true
I20260807 22:38:06.541970 15797 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f08c9562c918471ca62f4d1a6d172f76 in term 0.
I20260807 22:38:06.542255 15975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:38:06.542316 15975 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:38:06.542340 15975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:06.543313 15975 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.543431 15975 leader_election.cc:288] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [CANDIDATE]: Term 1 election: Requested vote from peers 8a315fe9f3da404a9d17e2db4368d24e (127.28.82.254:39851), 3963e1c6e0d94437ac43dd20b5bd1a69 (127.28.82.253:35257)
I20260807 22:38:06.543704 15691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08c9562c918471ca62f4d1a6d172f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a315fe9f3da404a9d17e2db4368d24e"
I20260807 22:38:06.543776 15691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:06.545030 15691 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08c9562c918471ca62f4d1a6d172f76 in term 1.
I20260807 22:38:06.545300 15881 leader_election.cc:302] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a315fe9f3da404a9d17e2db4368d24e, f08c9562c918471ca62f4d1a6d172f76; no voters:
I20260807 22:38:06.545435 15797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08c9562c918471ca62f4d1a6d172f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69"
I20260807 22:38:06.545524 15797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:06.546382 15797 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08c9562c918471ca62f4d1a6d172f76 in term 1.
I20260807 22:38:06.547469 15975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:38:06.547624 15975 raft_consensus.cc:686] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [term 1 LEADER]: Becoming Leader. State: Replica: f08c9562c918471ca62f4d1a6d172f76, State: Running, Role: LEADER
I20260807 22:38:06.547721 15975 consensus_queue.cc:237] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } }
I20260807 22:38:06.548074 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08c9562c918471ca62f4d1a6d172f76. Latest consensus state: current_term: 1 leader_uuid: "f08c9562c918471ca62f4d1a6d172f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.548137 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:06.548486 16049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:38:06.548735 16049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:38:06.550875 15797 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [term 1 FOLLOWER]: Refusing update from remote peer f08c9562c918471ca62f4d1a6d172f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:06.551097 15691 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [term 1 FOLLOWER]: Refusing update from remote peer f08c9562c918471ca62f4d1a6d172f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:06.551532 15975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:06.552815 16026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08c9562c918471ca62f4d1a6d172f76. Latest consensus state: current_term: 1 leader_uuid: "f08c9562c918471ca62f4d1a6d172f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.552897 16026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:38:06.552990 15657 tablet.cc:2369] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:38:06.553195 15975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:06.554435 16006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08c9562c918471ca62f4d1a6d172f76. Latest consensus state: current_term: 1 leader_uuid: "f08c9562c918471ca62f4d1a6d172f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.554543 16006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:38:06.556402 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f08c9562c918471ca62f4d1a6d172f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.556481 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:06.556990 16026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f08c9562c918471ca62f4d1a6d172f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.557066 16026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:38:06.557402 16050 mvcc.cc:205] Tried to move back new op lower bound from 7316038805709692928 to 7316038805700014080. Current Snapshot: MvccSnapshot[applied={T|T < 7316038805709692928}]
I20260807 22:38:06.557602 16053 mvcc.cc:205] Tried to move back new op lower bound from 7316038805709692928 to 7316038805700014080. Current Snapshot: MvccSnapshot[applied={T|T < 7316038805709692928}]
I20260807 22:38:06.558079 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f08c9562c918471ca62f4d1a6d172f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.558152 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:06.558712 16006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f08c9562c918471ca62f4d1a6d172f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a315fe9f3da404a9d17e2db4368d24e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39851 } } peers { permanent_uuid: "3963e1c6e0d94437ac43dd20b5bd1a69" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 35257 } } peers { permanent_uuid: "f08c9562c918471ca62f4d1a6d172f76" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41157 } } }
I20260807 22:38:06.558792 16006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:38:06.559860 16055 mvcc.cc:205] Tried to move back new op lower bound from 7316038805709692928 to 7316038805700014080. Current Snapshot: MvccSnapshot[applied={T|T < 7316038805709692928}]
I20260807 22:38:06.561249 16049 catalog_manager.cc:1347] Generated new cluster ID: 8b55bcec966c4f4c97507025ac3177f5
I20260807 22:38:06.561296 16049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:38:06.586086 16049 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:38:06.586980 16049 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:38:06.595217 16049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76: Generated new TSK 0
I20260807 22:38:06.595496 16049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:38:07.494432 16021 catalog_manager.cc:1259] Loaded cluster ID: 8b55bcec966c4f4c97507025ac3177f5
I20260807 22:38:07.494537 16021 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69: loading cluster ID for follower catalog manager: success
I20260807 22:38:07.495405 16021 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69: acquiring CA information for follower catalog manager: success
I20260807 22:38:07.495772 16021 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3963e1c6e0d94437ac43dd20b5bd1a69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:38:07.514340 16041 catalog_manager.cc:1259] Loaded cluster ID: 8b55bcec966c4f4c97507025ac3177f5
I20260807 22:38:07.514410 16041 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: loading cluster ID for follower catalog manager: success
I20260807 22:38:07.515292 16041 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: acquiring CA information for follower catalog manager: success
I20260807 22:38:07.515630 16041 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8a315fe9f3da404a9d17e2db4368d24e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:38:07.662207 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 15628
I20260807 22:38:07.678349 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 15741
I20260807 22:38:07.681375 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:07.681119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39851 (local address 127.0.0.1:52866)
0807 22:38:07.681262 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:07.681321 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:39851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":54,"threads_started":1}
W20260807 22:38:07.681725 15881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.254:39851: connect: Connection refused (error 111)
I20260807 22:38:07.682157 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:07.682066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39851 (local address 127.0.0.1:52870)
0807 22:38:07.682112 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:07.682132 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:39851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:38:07.682322 15881 consensus_peers.cc:487] T 00000000000000000000000000000000 P f08c9562c918471ca62f4d1a6d172f76 -> Peer 8a315fe9f3da404a9d17e2db4368d24e (127.28.82.254:39851): Couldn't send request to peer 8a315fe9f3da404a9d17e2db4368d24e. Status: Network error: Client connection negotiation failed: client connection to 127.28.82.254:39851: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 22:38:07.691239 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 15851
2026-08-07T22:38:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3154 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:38:07 dist-test-slave-7kzw krb5kdc[16088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:38:07 dist-test-slave-7kzw krb5kdc[16088](info): set up 2 sockets
Aug 07 22:38:07 dist-test-slave-7kzw krb5kdc[16088](info): commencing operation
krb5kdc: starting...
W20260807 22:38:08.900910 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:38:08 dist-test-slave-7kzw krb5kdc[16088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:38:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:38:09.029811 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37779
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:37779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:09.112751 16107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:09.113019 16107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:09.113055 16107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:09.114180 16107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:09.114253 16107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:09.114276 16107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:09.114297 16107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:09.114328 16107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:09.115835 16107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:37779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:09.116111 16107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:09.116359 16107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:38:09.121491 16107 server_base.cc:1034] running on GCE node
W20260807 22:38:09.121462 16116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:09.121459 16114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:09.121462 16119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:09.121968 16107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:09.122272 16107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:09.123447 16107 hybrid_clock.cc:648] HybridClock initialized: now 1786142289123417 us; error 59 us; skew 500 ppm
Aug 07 22:38:09 dist-test-slave-7kzw krb5kdc[16088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142289, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:09.125981 16107 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:38:09.126431 16107 webserver.cc:460] Webserver started at http://127.28.82.254:43611/ using document root <none> and password file <none>
I20260807 22:38:09.126688 16107 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:09.126734 16107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:09.126827 16107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:09.127862 16107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "8f9e26b401f6448d8125fad57a92b0d4"
format_stamp: "Formatted at 2026-08-07 22:38:09 on dist-test-slave-7kzw"
server_key: "02204bf492cbc80ab5d448d7f88b8cad"
server_key_iv: "36696e92288f87d81b8cdb85022e6b98"
server_key_version: "encryptionkey@0"
I20260807 22:38:09.128201 16107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "8f9e26b401f6448d8125fad57a92b0d4"
format_stamp: "Formatted at 2026-08-07 22:38:09 on dist-test-slave-7kzw"
server_key: "02204bf492cbc80ab5d448d7f88b8cad"
server_key_iv: "36696e92288f87d81b8cdb85022e6b98"
server_key_version: "encryptionkey@0"
I20260807 22:38:09.129509 16107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:38:09.130191 16129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:09.130390 16107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:38:09.130504 16107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "8f9e26b401f6448d8125fad57a92b0d4"
format_stamp: "Formatted at 2026-08-07 22:38:09 on dist-test-slave-7kzw"
server_key: "02204bf492cbc80ab5d448d7f88b8cad"
server_key_iv: "36696e92288f87d81b8cdb85022e6b98"
server_key_version: "encryptionkey@0"
I20260807 22:38:09.130584 16107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:09.148109 16107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:09.150411 16107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:09.150594 16107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:09.154690 16107 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:37779
I20260807 22:38:09.154758 16206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:37779 every 8 connection(s)
I20260807 22:38:09.155110 16107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:38:09.155635 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 16107
I20260807 22:38:09.155718 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:38:09.155943 29003 external_mini_cluster.cc:1428] Setting key 280a61deb8e1e2209ffe62fdd2a1a687
I20260807 22:38:09.156296 16207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:09.160055 16207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4: Bootstrap starting.
I20260807 22:38:09.160837 16207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:09.161186 16207 log.cc:824] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:09.162037 16207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4: No bootstrap required, opened a new log
Aug 07 22:38:09 dist-test-slave-7kzw krb5kdc[16088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:09.164395 16207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9e26b401f6448d8125fad57a92b0d4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37779 } }
I20260807 22:38:09.164649 16207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:09.164721 16207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9e26b401f6448d8125fad57a92b0d4, State: Initialized, Role: FOLLOWER
I20260807 22:38:09.164923 16207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9e26b401f6448d8125fad57a92b0d4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37779 } }
I20260807 22:38:09.165025 16207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:38:09.165066 16207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:38:09.165122 16207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:09.165670 16207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9e26b401f6448d8125fad57a92b0d4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37779 } }
I20260807 22:38:09.165800 16207 leader_election.cc:302] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f9e26b401f6448d8125fad57a92b0d4; no voters:
I20260807 22:38:09.165982 16207 leader_election.cc:288] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:38:09.166056 16214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:38:09.166208 16214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [term 1 LEADER]: Becoming Leader. State: Replica: 8f9e26b401f6448d8125fad57a92b0d4, State: Running, Role: LEADER
I20260807 22:38:09.166301 16207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:09.166332 16214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9e26b401f6448d8125fad57a92b0d4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37779 } }
I20260807 22:38:09.166785 16214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f9e26b401f6448d8125fad57a92b0d4. Latest consensus state: current_term: 1 leader_uuid: "8f9e26b401f6448d8125fad57a92b0d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9e26b401f6448d8125fad57a92b0d4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37779 } } }
I20260807 22:38:09.167016 16214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:09.168143 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:09.156482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.28.82.254:37779)
0807 22:38:09.156689 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:09.156699 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:09.156736 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:38:09.157311 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:09.157321 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:09.157388 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:09.157543 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:09.158138 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:09.159703 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:09.160491 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:09.161192 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:09.161317 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:09.163783 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:09.163790 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:09.163791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:09.163806 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:09.165767 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:09.166277 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:09.166280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:09.166280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:09.166395 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:09.166678 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:09.166679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:09.166680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:09.167033 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:09.167205 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:09.167717 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 22:38:09.167876 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
W20260807 22:38:09.168361 16238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:38:09.168416 16238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:38:09.166950 16215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f9e26b401f6448d8125fad57a92b0d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9e26b401f6448d8125fad57a92b0d4" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37779 } } }
I20260807 22:38:09.168534 16231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:38:09.168586 16215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:09.168812 16231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:38:09.169466 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:38:09.170631 16231 catalog_manager.cc:1347] Generated new cluster ID: 627c5f2478c84cdbbb4c639469ea37bd
I20260807 22:38:09.170691 16231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:38:09.185189 16231 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:38:09.185778 16231 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:38:09.187654 16231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f9e26b401f6448d8125fad57a92b0d4: Generated new TSK 0
I20260807 22:38:09.187841 16231 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:38:09.223790 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 16107
2026-08-07T22:38:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1533 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:38:09 dist-test-slave-7kzw krb5kdc[16249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:38:09 dist-test-slave-7kzw krb5kdc[16249](info): set up 2 sockets
Aug 07 22:38:09 dist-test-slave-7kzw krb5kdc[16249](info): commencing operation
krb5kdc: starting...
W20260807 22:38:10.448432 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:38:10 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:38:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:38:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:38:10.570940 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46285
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:46285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:10.656458 16265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:10.656697 16265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:10.656735 16265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:10.657898 16265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:10.657984 16265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:10.658017 16265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:10.658048 16265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:10.658072 16265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:10.659564 16265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:46285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:10.659838 16265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:10.660058 16265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:10.665010 16278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:10.665057 16265 server_base.cc:1034] running on GCE node
W20260807 22:38:10.665028 16274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:10.665037 16275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:10.665513 16265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:10.665855 16265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:10.667022 16265 hybrid_clock.cc:648] HybridClock initialized: now 1786142290667008 us; error 27 us; skew 500 ppm
Aug 07 22:38:10 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142290, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:10.673651 16265 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:38:10.674093 16265 webserver.cc:460] Webserver started at http://127.28.82.254:46537/ using document root <none> and password file <none>
I20260807 22:38:10.674332 16265 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:10.674381 16265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:10.674566 16265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:10.675691 16265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "78cc1a7446664928847c112519abaa47"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc252830bac296d8f6c0cb41b24046e4"
server_key_iv: "18667979c655b199780999cfdfac89c4"
server_key_version: "encryptionkey@0"
I20260807 22:38:10.676055 16265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "78cc1a7446664928847c112519abaa47"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc252830bac296d8f6c0cb41b24046e4"
server_key_iv: "18667979c655b199780999cfdfac89c4"
server_key_version: "encryptionkey@0"
I20260807 22:38:10.677421 16265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:38:10.678192 16292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:10.678406 16265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:10.678548 16265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "78cc1a7446664928847c112519abaa47"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc252830bac296d8f6c0cb41b24046e4"
server_key_iv: "18667979c655b199780999cfdfac89c4"
server_key_version: "encryptionkey@0"
I20260807 22:38:10.678634 16265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:10.706907 16265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:10.709009 16265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:10.709128 16265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:10.713173 16265 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:46285
I20260807 22:38:10.713279 16371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:46285 every 8 connection(s)
I20260807 22:38:10.713608 16265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:38:10.714725 16375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:10.715950 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 16265
I20260807 22:38:10.716040 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:38:10.716220 29003 external_mini_cluster.cc:1428] Setting key 960f021a90e8bcf2dceae16b986a6cce
I20260807 22:38:10.717604 16375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Bootstrap starting.
I20260807 22:38:10.718276 16375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:10.718669 16375 log.cc:824] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:10.719368 16375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: No bootstrap required, opened a new log
I20260807 22:38:10.721318 16375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:10.721477 16375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:10.721530 16375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78cc1a7446664928847c112519abaa47, State: Initialized, Role: FOLLOWER
I20260807 22:38:10.721684 16375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:10.721781 16375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:38:10.721827 16375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:38:10.721877 16375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:10.722450 16375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:10.722589 16375 leader_election.cc:302] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78cc1a7446664928847c112519abaa47; no voters:
I20260807 22:38:10.722766 16375 leader_election.cc:288] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:38:10.722831 16385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 1 FOLLOWER]: Leader election won for term 1
Aug 07 22:38:10 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:10.722987 16385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 1 LEADER]: Becoming Leader. State: Replica: 78cc1a7446664928847c112519abaa47, State: Running, Role: LEADER
I20260807 22:38:10.723073 16385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:10.723075 16375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:10.723524 16385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78cc1a7446664928847c112519abaa47. Latest consensus state: current_term: 1 leader_uuid: "78cc1a7446664928847c112519abaa47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } } }
I20260807 22:38:10.724042 16385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:10.723683 16386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78cc1a7446664928847c112519abaa47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } } }
I20260807 22:38:10.724633 16386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:10.724931 16407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:38:10.725541 16407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:38:10.727339 16407 catalog_manager.cc:1347] Generated new cluster ID: 60f71a10d2584db9a7eed4a2b3d24219
I20260807 22:38:10.727537 16407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:38:10.727705 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:10.716791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.28.82.254:46285)
0807 22:38:10.717005 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:10.717015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:10.717052 (+ 37us) server_negotiation.cc:407] Connection header received
0807 22:38:10.717583 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:10.717592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:10.717655 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:10.717808 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:10.718395 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:10.719393 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:10.720163 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:10.720578 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:10.720702 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:10.723645 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:10.723650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:10.723650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:10.723675 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:10.725841 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:10.726261 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:10.726264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:10.726265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:10.726366 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:10.726710 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:10.726711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:10.726711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:10.726996 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:10.727160 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:10.727299 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 22:38:10.727447 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":98,"threads_started":1}
I20260807 22:38:10.733251 16407 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:38:10.733762 16407 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:38:10.739627 16407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Generated new TSK 0
I20260807 22:38:10.739807 16407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:38:10.770344 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46285
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:10.857295 16415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:10.857525 16415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:10.857566 16415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:10.858777 16415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:10.858839 16415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:10.858882 16415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:38:10.860327 16415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:10.860591 16415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:10.860818 16415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:10.867585 16421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:10.867734 16415 server_base.cc:1034] running on GCE node
W20260807 22:38:10.867586 16424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:10.867592 16422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:10.868050 16415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:10.868252 16415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:10.869402 16415 hybrid_clock.cc:648] HybridClock initialized: now 1786142290869376 us; error 63 us; skew 500 ppm
Aug 07 22:38:10 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142290, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:10.871945 16415 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:38:10.872418 16415 webserver.cc:460] Webserver started at http://127.28.82.193:35341/ using document root <none> and password file <none>
I20260807 22:38:10.872646 16415 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:10.872715 16415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:10.872875 16415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:10.873915 16415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "d7233e3b3c694412a9b92680c588b74f"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc88297a11cf2d1f9f37126937e7eb37"
server_key_iv: "99bc28b260669f3ead376efbc329cb33"
server_key_version: "encryptionkey@0"
I20260807 22:38:10.874291 16415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "d7233e3b3c694412a9b92680c588b74f"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc88297a11cf2d1f9f37126937e7eb37"
server_key_iv: "99bc28b260669f3ead376efbc329cb33"
server_key_version: "encryptionkey@0"
I20260807 22:38:10.875865 16415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:38:10.876585 16434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:10.876766 16415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:38:10.876873 16415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "d7233e3b3c694412a9b92680c588b74f"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc88297a11cf2d1f9f37126937e7eb37"
server_key_iv: "99bc28b260669f3ead376efbc329cb33"
server_key_version: "encryptionkey@0"
I20260807 22:38:10.876958 16415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:10.889206 16415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:10.891012 16415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:10.891175 16415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:10.891407 16415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:10.891710 16415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:10.891773 16415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:10.891812 16415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:10.891840 16415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:10.898152 16415 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:43093
I20260807 22:38:10.898198 16585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:43093 every 8 connection(s)
I20260807 22:38:10.898535 16415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:38:10 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142290, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:10.906229 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 16415
I20260807 22:38:10.906365 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:38:10.906663 29003 external_mini_cluster.cc:1428] Setting key 96a203503be50735b51d38431dcdc11d
I20260807 22:38:10.908372 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:10.899184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:37583)
0807 22:38:10.899520 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:10.899571 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:10.900118 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:10.900372 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:10.900376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:10.900974 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:10.901166 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:10.901170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:10.901922 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:10.901923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:10.902762 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:10.902764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:10.903202 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:10.903203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:10.903331 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:10.903795 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:10.903803 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:10.905185 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:10.906965 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:10.906973 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:10.906975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:10.907210 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:10.907472 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:10.907473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:10.907474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:10.907589 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:10.907885 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:10.907890 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:10.908005 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 22:38:10.908089 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 22:38:10.908211 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":114,"threads_started":1}
I20260807 22:38:10.908392 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:10.899260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:37583 (local address 127.28.82.254:46285)
0807 22:38:10.899474 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:10.899478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:10.899685 (+ 207us) server_negotiation.cc:407] Connection header received
0807 22:38:10.900233 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:10.900235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:10.900248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:10.900306 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:10.901283 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:10.901816 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:10.902861 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:10.903090 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:10.903186 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:10.905286 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:10.905287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:10.905287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:10.905295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:10.906851 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:10.907318 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:10.907319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:10.907320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:10.907386 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:10.907660 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:10.907661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:10.907661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:10.907813 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:10.907892 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:10.908142 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 22:38:10.908247 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 22:38:10.908916 16588 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46285
I20260807 22:38:10.909025 16588 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:10.909229 16588 heartbeater.cc:511] Master 127.28.82.254:46285 requested a full tablet report, sending...
I20260807 22:38:10.909829 16317 ts_manager.cc:194] Registered new tserver with Master: d7233e3b3c694412a9b92680c588b74f (127.28.82.193:43093)
I20260807 22:38:10.910533 16317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:37583
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:38:10.934665 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46285
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:11.017182 16596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:11.017431 16596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:11.017473 16596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:11.018669 16596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:11.018745 16596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:11.018790 16596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:38:11.020301 16596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:11.020586 16596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:11.020826 16596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:11.026166 16607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.026208 16605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.026320 16596 server_base.cc:1034] running on GCE node
W20260807 22:38:11.026336 16604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.026664 16596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:11.026937 16596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:11.028105 16596 hybrid_clock.cc:648] HybridClock initialized: now 1786142291028088 us; error 33 us; skew 500 ppm
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:11.030771 16596 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:38:11.031152 16596 webserver.cc:460] Webserver started at http://127.28.82.194:34847/ using document root <none> and password file <none>
I20260807 22:38:11.031348 16596 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:11.031390 16596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:11.031481 16596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:11.032460 16596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "435ace9be1f84514ac8e01d7f6729ec1"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "bdf26c55bb16acca908285e443fe6e88"
server_key_iv: "2c1f00f25a6d3c9738ff95957404b6a0"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.032778 16596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "435ace9be1f84514ac8e01d7f6729ec1"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "bdf26c55bb16acca908285e443fe6e88"
server_key_iv: "2c1f00f25a6d3c9738ff95957404b6a0"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.034013 16596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:38:11.034729 16619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.034909 16596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:11.035019 16596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "435ace9be1f84514ac8e01d7f6729ec1"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "bdf26c55bb16acca908285e443fe6e88"
server_key_iv: "2c1f00f25a6d3c9738ff95957404b6a0"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.035097 16596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:11.046792 16596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:11.049135 16596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:11.049301 16596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:11.049538 16596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:11.049878 16596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:11.049934 16596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.049978 16596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:11.050005 16596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.056197 16596 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:39047
I20260807 22:38:11.056258 16783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:39047 every 8 connection(s)
I20260807 22:38:11.056564 16596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:38:11.059542 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 16596
I20260807 22:38:11.059660 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:38:11.059863 29003 external_mini_cluster.cc:1428] Setting key 97d8467f913c86e0baa8afce69d444a2
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:11.067199 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.057256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:44735 (local address 127.28.82.254:46285)
0807 22:38:11.057307 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.057311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.057565 (+ 254us) server_negotiation.cc:407] Connection header received
0807 22:38:11.058170 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.058171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.058183 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.058242 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:11.059276 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.060069 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.061110 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.061326 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.061405 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.064004 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:11.064005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:11.064005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:11.064014 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:11.065729 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.066139 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.066141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.066141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.066205 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.066459 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.066460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.066461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.066640 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:11.066721 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.066974 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.067062 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 22:38:11.067211 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.057253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:44735)
0807 22:38:11.057439 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.057475 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.057957 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.058337 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.058341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.058989 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:11.059182 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.059188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.060189 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.060190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.061013 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.061016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.061416 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.061418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.061563 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.062109 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:11.062117 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:11.063912 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:11.065831 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.065838 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.065840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.066060 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.066279 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.066280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.066280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.066389 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.066710 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:11.066716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:11.066833 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.066918 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.067041 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260807 22:38:11.067736 16787 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46285
I20260807 22:38:11.067852 16787 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:11.068050 16787 heartbeater.cc:511] Master 127.28.82.254:46285 requested a full tablet report, sending...
I20260807 22:38:11.068459 16317 ts_manager.cc:194] Registered new tserver with Master: 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047)
I20260807 22:38:11.068907 16317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:44735
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:38:11.089135 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46285
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:11.174333 16794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:11.174557 16794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:11.174580 16794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:11.175765 16794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:11.175839 16794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:11.175869 16794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:38:11.177325 16794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:11.177561 16794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:11.177785 16794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:11.183086 16800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.183097 16801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.183131 16803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.183382 16794 server_base.cc:1034] running on GCE node
I20260807 22:38:11.183540 16794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:11.183753 16794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:11.184922 16794 hybrid_clock.cc:648] HybridClock initialized: now 1786142291184905 us; error 32 us; skew 500 ppm
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:11.187467 16794 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:38:11.187966 16794 webserver.cc:460] Webserver started at http://127.28.82.195:38069/ using document root <none> and password file <none>
I20260807 22:38:11.188205 16794 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:11.188278 16794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:11.188431 16794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:11.189432 16794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "0a529469b3544087bb6952ac4424b2fd"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "85eabdfd553f7b539595186049ff8993"
server_key_iv: "2247dce6da87c247b3bf7d863ed165da"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.189807 16794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "0a529469b3544087bb6952ac4424b2fd"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "85eabdfd553f7b539595186049ff8993"
server_key_iv: "2247dce6da87c247b3bf7d863ed165da"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.191237 16794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:38:11.191942 16811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.192123 16794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:38:11.192230 16794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "0a529469b3544087bb6952ac4424b2fd"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "85eabdfd553f7b539595186049ff8993"
server_key_iv: "2247dce6da87c247b3bf7d863ed165da"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.192307 16794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:11.207811 16794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:11.209820 16794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:11.210000 16794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:11.210253 16794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:11.210635 16794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:11.210692 16794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.210736 16794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:11.210774 16794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.216974 16794 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:32983
I20260807 22:38:11.217023 16970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:32983 every 8 connection(s)
I20260807 22:38:11.217345 16794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:11.225144 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 16794
I20260807 22:38:11.225263 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:38:11.225487 29003 external_mini_cluster.cc:1428] Setting key afc097d77f155179bfbf324a63d5a3b9
I20260807 22:38:11.227349 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.218090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:49069 (local address 127.28.82.254:46285)
0807 22:38:11.218136 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.218138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.218322 (+ 184us) server_negotiation.cc:407] Connection header received
0807 22:38:11.218952 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.218954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.218968 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.219025 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:11.220002 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.220530 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.221522 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.221745 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.221837 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.224199 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:11.224201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:11.224201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:11.224210 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:11.226034 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.226525 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.226526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.226526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.226613 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.226843 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.226843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.226843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.226941 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:11.226988 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.227214 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.227286 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 22:38:11.227593 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.218034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:49069)
0807 22:38:11.218189 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.218235 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.218842 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.219093 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.219097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.219697 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:11.219899 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.219905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.220636 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.220637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.221428 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.221430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.221850 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.221851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.221980 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.222514 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:11.222524 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:11.224092 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:11.226133 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.226141 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.226144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.226401 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.226684 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.226684 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.226685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.226797 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.226994 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:11.226998 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:11.227108 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.227181 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.227322 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260807 22:38:11.228060 16973 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46285
I20260807 22:38:11.228183 16973 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:11.228363 16973 heartbeater.cc:511] Master 127.28.82.254:46285 requested a full tablet report, sending...
I20260807 22:38:11.228714 16317 ts_manager.cc:194] Registered new tserver with Master: 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
I20260807 22:38:11.229081 16317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:49069
I20260807 22:38:11.237625 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:38:11.244786 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.239118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.28.82.254:46285)
0807 22:38:11.239172 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.239173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.239272 (+ 99us) server_negotiation.cc:407] Connection header received
0807 22:38:11.239313 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.239313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.239321 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.239373 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:11.239974 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.240437 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.241145 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.241324 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.241374 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.242438 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:11.242439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:11.242439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:11.242445 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:11.243873 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.244174 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.244174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.244175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.244225 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.244391 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.244391 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.244392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.244511 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:11.244585 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.244670 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.244727 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 22:38:11.246256 16317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:38:11.247898 16317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:38:11.254832 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.250315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.28.82.194:39047)
0807 22:38:11.250801 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.250804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.250814 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:38:11.250852 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.250854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.250870 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.250944 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:11.251354 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.252140 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.253707 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.254394 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.254483 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.254548 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.254710 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.254755 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":120,"threads_started":1}
I20260807 22:38:11.254840 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.250205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:39047 (local address 127.0.0.1:57768)
0807 22:38:11.250563 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.250605 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.250670 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.251028 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.251029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.251033 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:11.251272 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.251274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.252251 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.252252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.253626 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.253629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.254505 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.254506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.254590 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.254607 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.254686 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.254757 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":248,"thread_start_us":52,"threads_started":1}
I20260807 22:38:11.254841 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.250543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43093 (local address 127.0.0.1:55222)
0807 22:38:11.251123 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.251132 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.251185 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.251441 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.251442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.251444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:11.251482 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.251483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.252644 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.252645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.253537 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.253540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.254527 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.254528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.254590 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.254607 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.254686 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.254752 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":4,"thread_start_us":73,"threads_started":1}
I20260807 22:38:11.255424 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.250655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55222 (local address 127.28.82.193:43093)
0807 22:38:11.250965 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.250969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.251182 (+ 213us) server_negotiation.cc:407] Connection header received
0807 22:38:11.251252 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.251254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.251266 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.251354 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:11.251569 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.252548 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.253648 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.254382 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.254484 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.254547 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.254974 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.255019 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":126,"threads_started":1}
I20260807 22:38:11.255725 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.250350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:32983 (local address 127.0.0.1:33434)
0807 22:38:11.250676 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.250687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.250749 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.251025 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.251028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.251030 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:11.251107 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.251110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.252345 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.252345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.253626 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.253629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.255475 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.255476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.255543 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.255547 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.255613 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.255651 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":47,"thread_start_us":83,"threads_started":1}
I20260807 22:38:11.255615 16683 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf50aa0538046c38fc398d73b2c95b5 (DEFAULT_TABLE table=test-table [id=2d89ec72071b428a97f75cef05f815c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:11.255975 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.250515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.28.82.195:32983)
0807 22:38:11.250705 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.250709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.250808 (+ 99us) server_negotiation.cc:407] Connection header received
0807 22:38:11.250852 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.250854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.250870 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.250944 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:11.251198 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.252247 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.254584 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.255311 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.255384 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.255438 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.255857 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.255899 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":82,"threads_started":1}
I20260807 22:38:11.256001 16683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf50aa0538046c38fc398d73b2c95b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:11.256058 16499 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf50aa0538046c38fc398d73b2c95b5 (DEFAULT_TABLE table=test-table [id=2d89ec72071b428a97f75cef05f815c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:11.256385 16499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf50aa0538046c38fc398d73b2c95b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:11.259488 16997 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Bootstrap starting.
I20260807 22:38:11.259914 17000 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Bootstrap starting.
I20260807 22:38:11.260201 16997 tablet_bootstrap.cc:654] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:11.260308 16883 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf50aa0538046c38fc398d73b2c95b5 (DEFAULT_TABLE table=test-table [id=2d89ec72071b428a97f75cef05f815c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:11.260607 17000 tablet_bootstrap.cc:654] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:11.260718 16883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf50aa0538046c38fc398d73b2c95b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:11.260788 16997 log.cc:824] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:11.261183 17000 log.cc:824] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:11.261674 16997 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: No bootstrap required, opened a new log
I20260807 22:38:11.261750 16997 ts_tablet_manager.cc:1397] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:11.263026 17000 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: No bootstrap required, opened a new log
I20260807 22:38:11.263110 17000 ts_tablet_manager.cc:1397] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:38:11.263841 16997 raft_consensus.cc:348] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.264029 16997 raft_consensus.cc:374] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:11.264084 16997 raft_consensus.cc:729] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435ace9be1f84514ac8e01d7f6729ec1, State: Initialized, Role: FOLLOWER
I20260807 22:38:11.264124 17005 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: Bootstrap starting.
I20260807 22:38:11.264218 16997 consensus_queue.cc:260] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.264541 16997 ts_tablet_manager.cc:1428] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:38:11.264541 16787 heartbeater.cc:503] Master 127.28.82.254:46285 was elected leader, sending a full tablet report...
I20260807 22:38:11.264810 17005 tablet_bootstrap.cc:654] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:11.264873 17000 raft_consensus.cc:348] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.265046 17000 raft_consensus.cc:374] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:11.265094 17000 raft_consensus.cc:729] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7233e3b3c694412a9b92680c588b74f, State: Initialized, Role: FOLLOWER
I20260807 22:38:11.265231 17000 consensus_queue.cc:260] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.265395 17005 log.cc:824] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:11.265579 16588 heartbeater.cc:503] Master 127.28.82.254:46285 was elected leader, sending a full tablet report...
I20260807 22:38:11.265842 17000 ts_tablet_manager.cc:1428] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:38:11.266335 17005 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: No bootstrap required, opened a new log
I20260807 22:38:11.266389 17005 ts_tablet_manager.cc:1397] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:11.267733 17005 raft_consensus.cc:348] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.267875 17005 raft_consensus.cc:374] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:11.267922 17005 raft_consensus.cc:729] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a529469b3544087bb6952ac4424b2fd, State: Initialized, Role: FOLLOWER
I20260807 22:38:11.268059 17005 consensus_queue.cc:260] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.268312 17005 ts_tablet_manager.cc:1428] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:11.268383 16973 heartbeater.cc:503] Master 127.28.82.254:46285 was elected leader, sending a full tablet report...
I20260807 22:38:11.306682 17009 raft_consensus.cc:482] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:11.306842 17009 raft_consensus.cc:504] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
W20260807 22:38:11.307165 16789 tablet.cc:2369] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:38:11.307307 17009 leader_election.cc:288] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), d7233e3b3c694412a9b92680c588b74f (127.28.82.193:43093)
I20260807 22:38:11.311209 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.307481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:39047 (local address 127.28.82.195:54589)
0807 22:38:11.307537 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.307553 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.307603 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.307802 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.307804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.307805 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:11.307886 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.307887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.308808 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.308809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.310013 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.310015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.310909 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.310910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.310996 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.310997 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.311062 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.311114 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":6}
I20260807 22:38:11.311223 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.307590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43093 (local address 127.28.82.195:34711)
0807 22:38:11.307825 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.307835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.307879 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.308092 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.308093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.308095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:11.308156 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.308158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.308966 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.308967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.310095 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.310097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.310909 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.310910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.310991 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.310996 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.311075 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.311113 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":5,"thread_start_us":129,"threads_started":1}
I20260807 22:38:11.311242 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.307567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:54589 (local address 127.28.82.194:39047)
0807 22:38:11.307613 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.307615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.307701 (+ 86us) server_negotiation.cc:407] Connection header received
0807 22:38:11.307731 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.307732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.307744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.307794 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:11.307965 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.308713 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.310157 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.310807 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.310899 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.310908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.311128 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.311176 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:38:11.311242 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.307807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:34711 (local address 127.28.82.193:43093)
0807 22:38:11.307860 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.307862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.307869 (+ 7us) server_negotiation.cc:407] Connection header received
0807 22:38:11.307970 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.307971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.307980 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.308047 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:11.308244 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.308885 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.310196 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.310806 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.310899 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.310908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.311128 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.311176 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:38:11.311575 16526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bf50aa0538046c38fc398d73b2c95b5" candidate_uuid: "0a529469b3544087bb6952ac4424b2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7233e3b3c694412a9b92680c588b74f" is_pre_election: true
I20260807 22:38:11.311641 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bf50aa0538046c38fc398d73b2c95b5" candidate_uuid: "0a529469b3544087bb6952ac4424b2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435ace9be1f84514ac8e01d7f6729ec1" is_pre_election: true
I20260807 22:38:11.311771 16526 raft_consensus.cc:2454] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a529469b3544087bb6952ac4424b2fd in term 0.
I20260807 22:38:11.311811 16711 raft_consensus.cc:2454] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a529469b3544087bb6952ac4424b2fd in term 0.
I20260807 22:38:11.312026 16813 leader_election.cc:302] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a529469b3544087bb6952ac4424b2fd, d7233e3b3c694412a9b92680c588b74f; no voters:
I20260807 22:38:11.312227 17009 raft_consensus.cc:2781] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:38:11.312317 17009 raft_consensus.cc:482] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:38:11.312340 17009 raft_consensus.cc:3037] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:11.313156 17009 raft_consensus.cc:504] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.313273 17009 leader_election.cc:288] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [CANDIDATE]: Term 1 election: Requested vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), d7233e3b3c694412a9b92680c588b74f (127.28.82.193:43093)
I20260807 22:38:11.313463 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bf50aa0538046c38fc398d73b2c95b5" candidate_uuid: "0a529469b3544087bb6952ac4424b2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435ace9be1f84514ac8e01d7f6729ec1"
I20260807 22:38:11.313506 16526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bf50aa0538046c38fc398d73b2c95b5" candidate_uuid: "0a529469b3544087bb6952ac4424b2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7233e3b3c694412a9b92680c588b74f"
I20260807 22:38:11.313555 16711 raft_consensus.cc:3037] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:11.313586 16526 raft_consensus.cc:3037] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:11.314525 16711 raft_consensus.cc:2454] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a529469b3544087bb6952ac4424b2fd in term 1.
I20260807 22:38:11.314579 16526 raft_consensus.cc:2454] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a529469b3544087bb6952ac4424b2fd in term 1.
I20260807 22:38:11.314837 16813 leader_election.cc:302] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a529469b3544087bb6952ac4424b2fd, d7233e3b3c694412a9b92680c588b74f; no voters:
I20260807 22:38:11.314940 17009 raft_consensus.cc:2781] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:38:11.315124 17009 raft_consensus.cc:686] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 1 LEADER]: Becoming Leader. State: Replica: 0a529469b3544087bb6952ac4424b2fd, State: Running, Role: LEADER
I20260807 22:38:11.315251 17009 consensus_queue.cc:237] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.316164 16317 catalog_manager.cc:5495] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a529469b3544087bb6952ac4424b2fd (127.28.82.195). New cstate: current_term: 1 leader_uuid: "0a529469b3544087bb6952ac4424b2fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } health_report { overall_health: UNKNOWN } } }
I20260807 22:38:11.327016 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.324577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33440 (local address 127.28.82.195:32983)
0807 22:38:11.324621 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.324624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.324633 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:38:11.324703 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.324704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.324722 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.324807 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:38:11.324945 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.325415 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.326190 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.326391 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.326522 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.326662 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:38:11.326763 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:38:11.326849 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.326900 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.326922 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:38:11.328831 16711 raft_consensus.cc:1261] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 1 FOLLOWER]: Refusing update from remote peer 0a529469b3544087bb6952ac4424b2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:11.328832 16526 raft_consensus.cc:1261] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Refusing update from remote peer 0a529469b3544087bb6952ac4424b2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:11.329232 17013 consensus_queue.cc:1035] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:11.329370 17009 consensus_queue.cc:1035] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:11.332055 17021 mvcc.cc:205] Tried to move back new op lower bound from 7316038825280196608 to 7316038825228083200. Current Snapshot: MvccSnapshot[applied={T|T < 7316038825280196608}]
I20260807 22:38:11.331878 17023 mvcc.cc:205] Tried to move back new op lower bound from 7316038825280196608 to 7316038825228083200. Current Snapshot: MvccSnapshot[applied={T|T < 7316038825280196608}]
I20260807 22:38:11.334229 16316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:11.334331 16316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:11.335116 16316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Sending DeleteTablet for 3 replicas of tablet 0bf50aa0538046c38fc398d73b2c95b5
I20260807 22:38:11.335479 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 16415
I20260807 22:38:11.335470 16683 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf50aa0538046c38fc398d73b2c95b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:11 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:57768
I20260807 22:38:11.335603 16499 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf50aa0538046c38fc398d73b2c95b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:11 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:55222
I20260807 22:38:11.336079 16883 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf50aa0538046c38fc398d73b2c95b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:11 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:33434
I20260807 22:38:11.336342 17036 tablet_replica.cc:330] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: stopping tablet replica
I20260807 22:38:11.336494 17036 raft_consensus.cc:2229] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:38:11.336676 17036 raft_consensus.cc:2258] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:11.340497 17036 ts_tablet_manager.cc:1905] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:11.343127 17036 ts_tablet_manager.cc:1918] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:11.343281 17036 log.cc:1196] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/0bf50aa0538046c38fc398d73b2c95b5
I20260807 22:38:11.343713 17036 ts_tablet_manager.cc:1939] T 0bf50aa0538046c38fc398d73b2c95b5 P 0a529469b3544087bb6952ac4424b2fd: Deleting consensus metadata
I20260807 22:38:11.344317 16295 catalog_manager.cc:4841] TS 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983): tablet 0bf50aa0538046c38fc398d73b2c95b5 (table test-table [id=2d89ec72071b428a97f75cef05f815c2]) successfully deleted
I20260807 22:38:11.347532 17033 tablet_replica.cc:330] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: stopping tablet replica
I20260807 22:38:11.347689 17033 raft_consensus.cc:2229] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:11.347842 17033 raft_consensus.cc:2258] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:11.348285 17033 ts_tablet_manager.cc:1905] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:11.349684 17033 ts_tablet_manager.cc:1918] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:11.349751 17033 log.cc:1196] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/0bf50aa0538046c38fc398d73b2c95b5
I20260807 22:38:11.350039 17033 ts_tablet_manager.cc:1939] T 0bf50aa0538046c38fc398d73b2c95b5 P 435ace9be1f84514ac8e01d7f6729ec1: Deleting consensus metadata
I20260807 22:38:11.350540 16295 catalog_manager.cc:4841] TS 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047): tablet 0bf50aa0538046c38fc398d73b2c95b5 (table test-table [id=2d89ec72071b428a97f75cef05f815c2]) successfully deleted
W20260807 22:38:11.352670 16295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:43093 (error 108)
I20260807 22:38:11.352921 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 16596
I20260807 22:38:11.356115 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.353113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43093 (local address 127.0.0.1:55236)
0807 22:38:11.356029 (+ 2916us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.356079 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2884}
W20260807 22:38:11.356298 16295 catalog_manager.cc:4584] TS d7233e3b3c694412a9b92680c588b74f (127.28.82.193:43093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0bf50aa0538046c38fc398d73b2c95b5: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43093: connect: Connection refused (error 111)
I20260807 22:38:11.368728 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 16794
I20260807 22:38:11.379174 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 16265
I20260807 22:38:11.390133 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46285
--webserver_interface=127.28.82.254
--webserver_port=46537
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:46285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:11.477427 17039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:11.477666 17039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:11.477703 17039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:11.478893 17039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:11.478961 17039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:11.478986 17039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:11.479005 17039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:11.479035 17039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:11.480485 17039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:46285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=46537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:11.480763 17039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:11.480984 17039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:11.490653 17051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.490653 17049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.490664 17048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.490969 17039 server_base.cc:1034] running on GCE node
I20260807 22:38:11.491281 17039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:11.491585 17039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:11.492733 17039 hybrid_clock.cc:648] HybridClock initialized: now 1786142291492720 us; error 29 us; skew 500 ppm
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:11.495389 17039 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:38:11.495838 17039 webserver.cc:460] Webserver started at http://127.28.82.254:46537/ using document root <none> and password file <none>
I20260807 22:38:11.496075 17039 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:11.496157 17039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:11.497540 17039 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:38:11.498221 17063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.498397 17039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:38:11.498528 17039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "78cc1a7446664928847c112519abaa47"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc252830bac296d8f6c0cb41b24046e4"
server_key_iv: "18667979c655b199780999cfdfac89c4"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.498942 17039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:11.508870 17039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:11.510654 17039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:11.510830 17039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:11.514876 17039 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:46285
I20260807 22:38:11.514907 17137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:46285 every 8 connection(s)
I20260807 22:38:11.515306 17039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:38:11.515664 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 17039
I20260807 22:38:11.516113 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:43093
--local_ip_for_outbound_sockets=127.28.82.193
--tserver_master_addrs=127.28.82.254:46285
--webserver_port=35341
--webserver_interface=127.28.82.193
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:11.517853 17138 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:38:11.518631 17138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Bootstrap starting.
I20260807 22:38:11.527933 17138 log.cc:824] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:11.529878 17138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:38:11.530197 17138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Bootstrap complete.
I20260807 22:38:11.532639 17138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:11.532860 17138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78cc1a7446664928847c112519abaa47, State: Initialized, Role: FOLLOWER
I20260807 22:38:11.533035 17138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:11.533097 17138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:38:11.533164 17138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:38:11.533244 17138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:38:11.534277 17138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:11.534364 17138 leader_election.cc:302] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78cc1a7446664928847c112519abaa47; no voters:
I20260807 22:38:11.534598 17138 leader_election.cc:288] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:38:11.534633 17144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:38:11.534758 17144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 2 LEADER]: Becoming Leader. State: Replica: 78cc1a7446664928847c112519abaa47, State: Running, Role: LEADER
I20260807 22:38:11.534879 17138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:11.534880 17144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:11.535300 17147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "78cc1a7446664928847c112519abaa47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } } }
I20260807 22:38:11.535310 17148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78cc1a7446664928847c112519abaa47. Latest consensus state: current_term: 2 leader_uuid: "78cc1a7446664928847c112519abaa47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } } }
I20260807 22:38:11.535470 17148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:11.535660 17147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:11.536168 17157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:38:11.536991 17157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:38:11.537101 17157 catalog_manager.cc:1259] Loaded cluster ID: 60f71a10d2584db9a7eed4a2b3d24219
I20260807 22:38:11.537140 17157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:38:11.537381 17157 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 22:38:11.541808 17157 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 22:38:11.544092 17157 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:38:11.544294 17157 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Loaded TSK: 0
I20260807 22:38:11.544500 17157 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:38:11.604606 17141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:11.604820 17141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:11.604844 17141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:11.606019 17141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:11.606060 17141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:11.606087 17141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:38:11.607664 17141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:43093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=35341
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:11.607918 17141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:11.608188 17141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:11.614187 17177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.614209 17173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.614432 17141 server_base.cc:1034] running on GCE node
W20260807 22:38:11.614513 17174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.614717 17141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:11.614969 17141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:11.616140 17141 hybrid_clock.cc:648] HybridClock initialized: now 1786142291616122 us; error 44 us; skew 500 ppm
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:11.619000 17141 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:38:11.619498 17141 webserver.cc:460] Webserver started at http://127.28.82.193:35341/ using document root <none> and password file <none>
I20260807 22:38:11.619735 17141 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:11.619827 17141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:11.621305 17141 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:11.622078 17189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.622344 17141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:11.622450 17141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "d7233e3b3c694412a9b92680c588b74f"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc88297a11cf2d1f9f37126937e7eb37"
server_key_iv: "99bc28b260669f3ead376efbc329cb33"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.622931 17141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:11.647259 17141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:11.650007 17141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:11.650182 17141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:11.650501 17141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:11.651052 17201 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 22:38:11.652642 17141 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:38:11.652715 17141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 22:38:11.652751 17141 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:38:11.653626 17141 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:38:11.653678 17141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 22:38:11.653755 17201 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Bootstrap starting.
I20260807 22:38:11.665825 17141 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:43093
I20260807 22:38:11.666352 17141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:38:11.668735 17201 log.cc:824] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:11.671850 17201 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:38:11.672155 17201 tablet_bootstrap.cc:492] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Bootstrap complete.
I20260807 22:38:11.672608 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 17141
I20260807 22:38:11.672907 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:39047
--local_ip_for_outbound_sockets=127.28.82.194
--tserver_master_addrs=127.28.82.254:46285
--webserver_port=34847
--webserver_interface=127.28.82.194
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:11.673555 17201 ts_tablet_manager.cc:1397] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.005s
I20260807 22:38:11.674642 17371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:43093 every 8 connection(s)
I20260807 22:38:11.675190 17201 raft_consensus.cc:348] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.675521 17201 raft_consensus.cc:729] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7233e3b3c694412a9b92680c588b74f, State: Initialized, Role: FOLLOWER
I20260807 22:38:11.675686 17201 consensus_queue.cc:260] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.675968 17201 ts_tablet_manager.cc:1428] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:11.687374 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.667237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:48469)
0807 22:38:11.669034 (+ 1797us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.669082 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.669799 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.676000 (+ 6201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.676005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.676384 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:11.676580 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.676585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.678241 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.678242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.679469 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.679472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.680373 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.680375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.680520 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.681179 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:11.681198 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:11.683143 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:11.685390 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.685399 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.685402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.685762 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.686073 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.686074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.686074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.686227 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.686724 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:11.686732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:11.686894 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.686999 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.687159 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1752,"thread_start_us":121,"threads_started":1}
I20260807 22:38:11.687392 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.668157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:48469 (local address 127.28.82.254:46285)
0807 22:38:11.668406 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.668416 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.675178 (+ 6762us) server_negotiation.cc:407] Connection header received
0807 22:38:11.675481 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.675489 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.675708 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.676687 (+ 979us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:11.676762 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.678130 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.679568 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.680273 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.680398 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.683261 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:11.683265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:11.683266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:11.683278 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:11.685293 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.685874 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.685877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.685878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.685991 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.686322 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.686324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.686324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.686642 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:11.686730 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.687123 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.687254 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":130,"threads_started":1}
I20260807 22:38:11.688097 17372 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46285
I20260807 22:38:11.688210 17372 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:11.688385 17372 heartbeater.cc:511] Master 127.28.82.254:46285 requested a full tablet report, sending...
I20260807 22:38:11.689163 17082 ts_manager.cc:194] Registered new tserver with Master: d7233e3b3c694412a9b92680c588b74f (127.28.82.193:43093)
I20260807 22:38:11.690330 17082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:48469
I20260807 22:38:11.691416 17372 heartbeater.cc:503] Master 127.28.82.254:46285 was elected leader, sending a full tablet report...
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:38:11.699330 17381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.689844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43093 (local address 127.0.0.1:55238)
0807 22:38:11.690061 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.690100 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.690181 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.690830 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.690836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.691219 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:11.691279 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.691282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.692571 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.692572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.693205 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.693208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.693882 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.693883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.693944 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.694447 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:11.694452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:11.696194 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:11.698146 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.698153 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.698155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.698357 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.698603 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.698604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.698604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.698668 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.698986 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:11.698992 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:11.699033 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.699171 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.699262 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":143,"threads_started":1}
I20260807 22:38:11.699409 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.689906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.28.82.193:43093)
0807 22:38:11.690655 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.690659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.690670 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:38:11.690703 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.690705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.690719 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.690804 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:11.691381 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.692469 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.693301 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.693782 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.693856 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.696290 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:11.696298 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:11.696299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:11.696306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:11.698038 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.698442 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.698445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.698446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.698534 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.698733 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.698734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.698735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.698913 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:11.699001 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.699211 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.699317 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":94,"threads_started":1}
I20260807 22:38:11.699682 17263 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf50aa0538046c38fc398d73b2c95b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:11 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:55238
I20260807 22:38:11.700333 17385 tablet_replica.cc:330] stopping tablet replica
I20260807 22:38:11.700444 17385 raft_consensus.cc:2229] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:11.700519 17385 raft_consensus.cc:2258] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:11.700847 17385 ts_tablet_manager.cc:1905] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:11.702914 17385 ts_tablet_manager.cc:1918] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:11.702979 17385 log.cc:1196] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/0bf50aa0538046c38fc398d73b2c95b5
I20260807 22:38:11.703227 17385 ts_tablet_manager.cc:1939] T 0bf50aa0538046c38fc398d73b2c95b5 P d7233e3b3c694412a9b92680c588b74f: Deleting consensus metadata
I20260807 22:38:11.703737 17065 catalog_manager.cc:4841] TS d7233e3b3c694412a9b92680c588b74f (127.28.82.193:43093): tablet 0bf50aa0538046c38fc398d73b2c95b5 (table test-table [id=2d89ec72071b428a97f75cef05f815c2]) successfully deleted
W20260807 22:38:11.765072 17377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:11.765414 17377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:11.765458 17377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:11.767084 17377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:11.767153 17377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:11.767199 17377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:38:11.768689 17377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:39047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=34847
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:11.768963 17377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:11.769209 17377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:11.774547 17391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.774682 17377 server_base.cc:1034] running on GCE node
W20260807 22:38:11.774569 17392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.774798 17395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.775005 17377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:11.775295 17377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:11.776482 17377 hybrid_clock.cc:648] HybridClock initialized: now 1786142291776452 us; error 42 us; skew 500 ppm
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:11.779245 17377 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:38:11.779690 17377 webserver.cc:460] Webserver started at http://127.28.82.194:34847/ using document root <none> and password file <none>
I20260807 22:38:11.779937 17377 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:11.780025 17377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:11.781431 17377 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:11.782068 17404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.782253 17377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.782361 17377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "435ace9be1f84514ac8e01d7f6729ec1"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "bdf26c55bb16acca908285e443fe6e88"
server_key_iv: "2c1f00f25a6d3c9738ff95957404b6a0"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.782778 17377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:11.795748 17377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:11.797753 17377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:11.797916 17377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:11.798147 17377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:11.798511 17377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:11.798571 17377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.798614 17377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:11.798641 17377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.804646 17377 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:39047
I20260807 22:38:11.804728 17559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:39047 every 8 connection(s)
I20260807 22:38:11.805069 17377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:38:11.810992 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 17377
I20260807 22:38:11.811319 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:32983
--local_ip_for_outbound_sockets=127.28.82.195
--tserver_master_addrs=127.28.82.254:46285
--webserver_port=38069
--webserver_interface=127.28.82.195
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:11.815457 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.805833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:53693 (local address 127.28.82.254:46285)
0807 22:38:11.805910 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.805912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.806111 (+ 199us) server_negotiation.cc:407] Connection header received
0807 22:38:11.806660 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.806661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.806673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.806739 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:11.807736 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.808233 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.809207 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.809430 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.809501 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.812231 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:11.812234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:11.812234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:11.812243 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:11.813818 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.814306 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.814307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.814308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.814379 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.814694 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.814695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.814696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.814828 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:11.814917 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.815232 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.815341 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 22:38:11.815743 17562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.805811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:53693)
0807 22:38:11.805978 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.806021 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.806574 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.806798 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.806802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.807395 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:11.807588 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.807592 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.808320 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.808321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.809104 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.809106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.809520 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.809521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.809698 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.810286 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:11.810295 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:11.812113 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:11.813931 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.813940 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.813943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.814199 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.814458 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.814459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.814460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.814612 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.814909 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:11.814917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:11.815075 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.815190 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.815362 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260807 22:38:11.816481 17560 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46285
I20260807 22:38:11.816586 17560 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:11.816795 17560 heartbeater.cc:511] Master 127.28.82.254:46285 requested a full tablet report, sending...
I20260807 22:38:11.817278 17082 ts_manager.cc:194] Registered new tserver with Master: 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047)
I20260807 22:38:11.817836 17082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:53693
W20260807 22:38:11.897671 17563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:11.897881 17563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:11.897904 17563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:11.899087 17563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:11.899132 17563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:11.899159 17563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:38:11.900638 17563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:32983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=38069
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:11.900893 17563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:11.901180 17563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:11.906368 17575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.906391 17573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:11.906363 17571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:11.906553 17563 server_base.cc:1034] running on GCE node
I20260807 22:38:11.906913 17563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:11.907124 17563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:11.908280 17563 hybrid_clock.cc:648] HybridClock initialized: now 1786142291908265 us; error 28 us; skew 500 ppm
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:11.910995 17563 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:38:11.911449 17563 webserver.cc:460] Webserver started at http://127.28.82.195:38069/ using document root <none> and password file <none>
I20260807 22:38:11.911690 17563 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:11.911751 17563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:11.913121 17563 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:11.913754 17586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.913947 17563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:38:11.914050 17563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "0a529469b3544087bb6952ac4424b2fd"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "85eabdfd553f7b539595186049ff8993"
server_key_iv: "2247dce6da87c247b3bf7d863ed165da"
server_key_version: "encryptionkey@0"
I20260807 22:38:11.914397 17563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:11.930356 17563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:11.932428 17563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:11.932606 17563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:11.932853 17563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:11.933174 17563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:11.933230 17563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.933272 17563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:11.933300 17563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.940137 17563 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:32983
I20260807 22:38:11.940182 17750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:32983 every 8 connection(s)
I20260807 22:38:11.940521 17563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:38:11 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:11.947523 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 17563
I20260807 22:38:11.949785 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.941272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:40475 (local address 127.28.82.254:46285)
0807 22:38:11.941358 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.941360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.941571 (+ 211us) server_negotiation.cc:407] Connection header received
0807 22:38:11.942032 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.942033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.942043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.942097 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:11.943081 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.943570 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.944513 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.944702 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.944752 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.946878 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:11.946880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:11.946880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:11.946888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:11.948402 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.948811 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.948812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.948812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.948875 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.949137 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.949138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.949138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.949255 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:11.949334 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.949606 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.949696 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:38:11.949975 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.941263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:40475)
0807 22:38:11.941444 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.941479 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.941948 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.942154 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.942157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.942794 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:11.942981 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.942986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.943678 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.943679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.944437 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.944440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.944776 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.944777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.944876 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.945320 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:11.945328 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:11.946781 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:11.948497 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.948504 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.948506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.948733 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.948951 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:11.948952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:11.948952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:11.949061 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:11.949325 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:11.949333 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:11.949472 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.949583 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.949737 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":119,"threads_started":1}
I20260807 22:38:11.950459 17751 heartbeater.cc:347] Connected to a master server at 127.28.82.254:46285
I20260807 22:38:11.950596 17751 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:11.950779 17751 heartbeater.cc:511] Master 127.28.82.254:46285 requested a full tablet report, sending...
I20260807 22:38:11.951243 17082 ts_manager.cc:194] Registered new tserver with Master: 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
I20260807 22:38:11.951609 17082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:40475
I20260807 22:38:11.954023 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.947959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.28.82.254:46285)
0807 22:38:11.948073 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.948075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.948111 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:38:11.948190 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.948191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.948203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.948281 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:11.948848 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.949317 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.950188 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.950444 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.950548 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.951862 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:11.951863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:11.951863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:11.951869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:11.953168 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.953478 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.953478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.953479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.953521 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:11.953677 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:11.953678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:11.953678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:11.953761 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:11.953883 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.953909 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.953963 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260807 22:38:11.954823 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:38:11.957268 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.955192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.28.82.254:46285)
0807 22:38:11.955241 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.955243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.955332 (+ 89us) server_negotiation.cc:407] Connection header received
0807 22:38:11.955376 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.955377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.955408 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.955462 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:38:11.955536 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.955953 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.956663 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.956851 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.956901 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.957031 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:38:11.957112 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:38:11.957167 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.957206 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.957224 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 22:38:11.957533 17082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:38:11.959232 17082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:38:11.961588 17263 tablet_service.cc:1467] Processing CreateTablet for tablet b2b44c35bf7c46848e215b983d07f334 (DEFAULT_TABLE table=test-table [id=92e7633236ac404eb3d9608f1b210fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:11.961848 17263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b44c35bf7c46848e215b983d07f334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:11.963604 17201 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Bootstrap starting.
I20260807 22:38:11.963932 17201 tablet_bootstrap.cc:654] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:11.964865 17201 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: No bootstrap required, opened a new log
I20260807 22:38:11.964923 17201 ts_tablet_manager.cc:1397] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:11.965077 17201 raft_consensus.cc:348] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.965167 17201 raft_consensus.cc:374] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:11.965221 17201 raft_consensus.cc:729] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7233e3b3c694412a9b92680c588b74f, State: Initialized, Role: FOLLOWER
I20260807 22:38:11.965268 17201 consensus_queue.cc:260] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.965353 17381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.961063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:39047 (local address 127.0.0.1:57782)
0807 22:38:11.961132 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.961146 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.961203 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.961567 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.961569 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.961571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:11.961608 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.961610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.962977 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.962978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.964023 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.964027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.965073 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.965074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.965146 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.965150 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.965199 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.965242 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":11}
I20260807 22:38:11.965376 17201 ts_tablet_manager.cc:1428] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:11.965467 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.961188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.28.82.194:39047)
0807 22:38:11.961366 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.961371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.961382 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:38:11.961425 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.961427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.961444 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.961560 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:11.961669 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.962838 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.964318 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.965001 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.965073 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.965127 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.965320 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.965368 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260807 22:38:11.965270 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.961161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:32983 (local address 127.0.0.1:33452)
0807 22:38:11.961301 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:11.961311 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:11.961358 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:11.961762 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:11.961763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:11.961766 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:11.961814 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.961816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.963047 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.963048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.963967 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:11.963971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.964995 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.964996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:11.965058 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.965081 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:38:11.965135 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 22:38:11.965180 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260807 22:38:11.965731 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:11.961403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.28.82.195:32983)
0807 22:38:11.961553 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:11.961558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:11.961568 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:38:11.961603 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:11.961605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:11.961620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:11.961697 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:11.961904 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.962946 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.964080 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:11.964821 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:11.964891 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:11.964951 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:11.965614 (+ 663us) server_negotiation.cc:299] Negotiation successful
0807 22:38:11.965659 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260807 22:38:11.966172 17474 tablet_service.cc:1467] Processing CreateTablet for tablet b2b44c35bf7c46848e215b983d07f334 (DEFAULT_TABLE table=test-table [id=92e7633236ac404eb3d9608f1b210fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:11.966454 17474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b44c35bf7c46848e215b983d07f334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:11.969214 17767 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Bootstrap starting.
I20260807 22:38:11.969756 17659 tablet_service.cc:1467] Processing CreateTablet for tablet b2b44c35bf7c46848e215b983d07f334 (DEFAULT_TABLE table=test-table [id=92e7633236ac404eb3d9608f1b210fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:11.969978 17767 tablet_bootstrap.cc:654] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:11.970084 17659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b44c35bf7c46848e215b983d07f334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:11.970573 17767 log.cc:824] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:11.971295 17767 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: No bootstrap required, opened a new log
I20260807 22:38:11.971381 17767 ts_tablet_manager.cc:1397] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:11.972927 17770 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: Bootstrap starting.
I20260807 22:38:11.972887 17767 raft_consensus.cc:348] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.973062 17767 raft_consensus.cc:374] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:11.973114 17767 raft_consensus.cc:729] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435ace9be1f84514ac8e01d7f6729ec1, State: Initialized, Role: FOLLOWER
I20260807 22:38:11.973253 17767 consensus_queue.cc:260] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.973567 17767 ts_tablet_manager.cc:1428] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:11.973588 17560 heartbeater.cc:503] Master 127.28.82.254:46285 was elected leader, sending a full tablet report...
I20260807 22:38:11.973781 17770 tablet_bootstrap.cc:654] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:11.974397 17770 log.cc:824] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:11.975117 17770 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: No bootstrap required, opened a new log
I20260807 22:38:11.975180 17770 ts_tablet_manager.cc:1397] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:11.976475 17770 raft_consensus.cc:348] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.976595 17770 raft_consensus.cc:374] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:11.976619 17770 raft_consensus.cc:729] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a529469b3544087bb6952ac4424b2fd, State: Initialized, Role: FOLLOWER
I20260807 22:38:11.976780 17770 consensus_queue.cc:260] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:11.977047 17770 ts_tablet_manager.cc:1428] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:11.977080 17751 heartbeater.cc:503] Master 127.28.82.254:46285 was elected leader, sending a full tablet report...
W20260807 22:38:12.055655 17561 tablet.cc:2369] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:38:12.073035 17378 raft_consensus.cc:482] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:12.073185 17378 raft_consensus.cc:504] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:12.073621 17378 leader_election.cc:288] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
I20260807 22:38:12.076912 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.074038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:40001 (local address 127.28.82.195:32983)
0807 22:38:12.074114 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:12.074116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:12.074280 (+ 164us) server_negotiation.cc:407] Connection header received
0807 22:38:12.074315 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:12.074316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:12.074333 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:12.074404 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:12.074555 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.075043 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.075975 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.076536 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.076613 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:12.076621 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:12.076826 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 22:38:12.076857 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:38:12.076983 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.073801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:39047 (local address 127.28.82.193:52569)
0807 22:38:12.073861 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.073874 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:12.073932 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:12.074235 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:12.074236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:12.074239 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:12.074325 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:12.074328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.075124 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.075125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:12.075854 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:12.075855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.076634 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.076635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:12.076722 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:12.076753 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 22:38:12.076814 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 22:38:12.076869 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":18}
I20260807 22:38:12.076990 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.073929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:32983 (local address 127.28.82.193:40001)
0807 22:38:12.074162 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.074176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:12.074254 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:12.074410 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:12.074411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:12.074413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:12.074491 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:12.074493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.075136 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.075137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:12.075883 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:12.075885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.076655 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.076656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:12.076722 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:12.076753 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 22:38:12.076814 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 22:38:12.076869 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":10,"thread_start_us":151,"threads_started":1}
I20260807 22:38:12.077214 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.073949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:52569 (local address 127.28.82.194:39047)
0807 22:38:12.074004 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:12.074006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:12.074017 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:38:12.074052 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:12.074053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:12.074064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:12.074260 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:12.074412 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.075010 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.075948 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.076518 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.076586 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:12.076596 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:12.077082 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 22:38:12.077134 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 22:38:12.077414 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b44c35bf7c46848e215b983d07f334" candidate_uuid: "d7233e3b3c694412a9b92680c588b74f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a529469b3544087bb6952ac4424b2fd" is_pre_election: true
I20260807 22:38:12.077466 17496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b44c35bf7c46848e215b983d07f334" candidate_uuid: "d7233e3b3c694412a9b92680c588b74f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435ace9be1f84514ac8e01d7f6729ec1" is_pre_election: true
I20260807 22:38:12.077593 17686 raft_consensus.cc:2454] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7233e3b3c694412a9b92680c588b74f in term 0.
I20260807 22:38:12.077623 17496 raft_consensus.cc:2454] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7233e3b3c694412a9b92680c588b74f in term 0.
I20260807 22:38:12.077836 17192 leader_election.cc:302] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 435ace9be1f84514ac8e01d7f6729ec1, d7233e3b3c694412a9b92680c588b74f; no voters:
I20260807 22:38:12.078009 17378 raft_consensus.cc:2781] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:38:12.078088 17378 raft_consensus.cc:482] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:38:12.078106 17378 raft_consensus.cc:3037] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:12.078915 17378 raft_consensus.cc:504] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:12.079056 17378 leader_election.cc:288] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 1 election: Requested vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
I20260807 22:38:12.079309 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b44c35bf7c46848e215b983d07f334" candidate_uuid: "d7233e3b3c694412a9b92680c588b74f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a529469b3544087bb6952ac4424b2fd"
I20260807 22:38:12.079290 17496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b44c35bf7c46848e215b983d07f334" candidate_uuid: "d7233e3b3c694412a9b92680c588b74f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435ace9be1f84514ac8e01d7f6729ec1"
I20260807 22:38:12.079388 17686 raft_consensus.cc:3037] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:12.079388 17496 raft_consensus.cc:3037] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:12.080348 17686 raft_consensus.cc:2454] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7233e3b3c694412a9b92680c588b74f in term 1.
I20260807 22:38:12.080348 17496 raft_consensus.cc:2454] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7233e3b3c694412a9b92680c588b74f in term 1.
I20260807 22:38:12.080560 17192 leader_election.cc:302] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 435ace9be1f84514ac8e01d7f6729ec1, d7233e3b3c694412a9b92680c588b74f; no voters:
I20260807 22:38:12.080709 17378 raft_consensus.cc:2781] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:38:12.080873 17378 raft_consensus.cc:686] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 LEADER]: Becoming Leader. State: Replica: d7233e3b3c694412a9b92680c588b74f, State: Running, Role: LEADER
I20260807 22:38:12.080963 17378 consensus_queue.cc:237] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:12.081725 17082 catalog_manager.cc:5495] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f reported cstate change: term changed from 0 to 1, leader changed from <none> to d7233e3b3c694412a9b92680c588b74f (127.28.82.193). New cstate: current_term: 1 leader_uuid: "d7233e3b3c694412a9b92680c588b74f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } health_report { overall_health: HEALTHY } } }
I20260807 22:38:12.109869 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.107618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.28.82.193:43093)
0807 22:38:12.107744 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:12.107746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:12.107755 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:38:12.107807 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:12.107808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:12.107827 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:12.107894 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:38:12.108004 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.108502 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.109189 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:12.109393 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:12.109449 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:12.109578 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:38:12.109689 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:38:12.109746 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:12.109784 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 22:38:12.109804 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 22:38:12.111491 17496 raft_consensus.cc:1261] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 1 FOLLOWER]: Refusing update from remote peer d7233e3b3c694412a9b92680c588b74f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:12.111491 17686 raft_consensus.cc:1261] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 1 FOLLOWER]: Refusing update from remote peer d7233e3b3c694412a9b92680c588b74f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:12.111900 17779 consensus_queue.cc:1035] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:12.112040 17378 consensus_queue.cc:1035] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [LEADER]: Connected to new peer: Peer: permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:12.114522 17784 mvcc.cc:205] Tried to move back new op lower bound from 7316038828486008832 to 7316038828364206080. Current Snapshot: MvccSnapshot[applied={T|T < 7316038828486008832}]
I20260807 22:38:12.115343 17785 mvcc.cc:205] Tried to move back new op lower bound from 7316038828486008832 to 7316038828364206080. Current Snapshot: MvccSnapshot[applied={T|T < 7316038828486008832}]
I20260807 22:38:12.116858 17082 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:12.116961 17082 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:12.117756 17082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Sending DeleteTablet for 3 replicas of tablet b2b44c35bf7c46848e215b983d07f334
I20260807 22:38:12.118069 17474 tablet_service.cc:1514] Processing DeleteTablet for tablet b2b44c35bf7c46848e215b983d07f334 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:12 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:57782
I20260807 22:38:12.118117 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 17141
I20260807 22:38:12.124351 17659 tablet_service.cc:1514] Processing DeleteTablet for tablet b2b44c35bf7c46848e215b983d07f334 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:12 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:33452
I20260807 22:38:12.124547 17800 tablet_replica.cc:330] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: stopping tablet replica
I20260807 22:38:12.124614 17801 tablet_replica.cc:330] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: stopping tablet replica
I20260807 22:38:12.124656 17800 raft_consensus.cc:2229] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:12.124742 17801 raft_consensus.cc:2229] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:12.124783 17800 raft_consensus.cc:2258] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:12.124853 17801 raft_consensus.cc:2258] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:12.125233 17800 ts_tablet_manager.cc:1905] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:12.125495 17801 ts_tablet_manager.cc:1905] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:12.126961 17800 ts_tablet_manager.cc:1918] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:12.127075 17800 log.cc:1196] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/b2b44c35bf7c46848e215b983d07f334
I20260807 22:38:12.127344 17801 ts_tablet_manager.cc:1918] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:12.127434 17801 log.cc:1196] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/b2b44c35bf7c46848e215b983d07f334
I20260807 22:38:12.127429 17800 ts_tablet_manager.cc:1939] T b2b44c35bf7c46848e215b983d07f334 P 435ace9be1f84514ac8e01d7f6729ec1: Deleting consensus metadata
I20260807 22:38:12.127692 17801 ts_tablet_manager.cc:1939] T b2b44c35bf7c46848e215b983d07f334 P 0a529469b3544087bb6952ac4424b2fd: Deleting consensus metadata
I20260807 22:38:12.128104 17065 catalog_manager.cc:4841] TS 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983): tablet b2b44c35bf7c46848e215b983d07f334 (table test-table [id=92e7633236ac404eb3d9608f1b210fc6]) successfully deleted
I20260807 22:38:12.130617 17065 catalog_manager.cc:4841] TS 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047): tablet b2b44c35bf7c46848e215b983d07f334 (table test-table [id=92e7633236ac404eb3d9608f1b210fc6]) successfully deleted
W20260807 22:38:12.131812 17065 connection.cc:535] client connection to 127.28.82.193:43093 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 22:38:12.131973 17065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:38:12.132264 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 17377
I20260807 22:38:12.133958 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.132327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43093 (local address 127.0.0.1:55262)
0807 22:38:12.133877 (+ 1550us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.133919 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1528}
W20260807 22:38:12.134094 17065 catalog_manager.cc:4584] TS d7233e3b3c694412a9b92680c588b74f (127.28.82.193:43093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2b44c35bf7c46848e215b983d07f334: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43093: connect: Connection refused (error 111)
I20260807 22:38:12.142853 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 17563
I20260807 22:38:12.154690 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 17039
I20260807 22:38:12.168078 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46285
--webserver_interface=127.28.82.254
--webserver_port=46537
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:46285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:12.251292 17803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:12.251499 17803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:12.251521 17803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:12.252591 17803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:12.252626 17803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:12.252640 17803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:12.252653 17803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:12.252665 17803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:12.254211 17803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:46285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:46285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=46537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:12.254406 17803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:12.254647 17803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:12.259727 17810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:12.259735 17815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:12.259821 17811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:12.259909 17803 server_base.cc:1034] running on GCE node
I20260807 22:38:12.260170 17803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:12.260461 17803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:12.261677 17803 hybrid_clock.cc:648] HybridClock initialized: now 1786142292261664 us; error 48 us; skew 500 ppm
Aug 07 22:38:12 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142292, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:12.264300 17803 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:38:12.264824 17803 webserver.cc:460] Webserver started at http://127.28.82.254:46537/ using document root <none> and password file <none>
I20260807 22:38:12.265050 17803 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:12.265118 17803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:12.266707 17803 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:12.267417 17824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:12.267609 17803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:12.267721 17803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "78cc1a7446664928847c112519abaa47"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc252830bac296d8f6c0cb41b24046e4"
server_key_iv: "18667979c655b199780999cfdfac89c4"
server_key_version: "encryptionkey@0"
I20260807 22:38:12.268096 17803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:12.280230 17803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:12.282375 17803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:12.282574 17803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:12.286796 17803 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:46285
I20260807 22:38:12.286846 17893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:46285 every 8 connection(s)
I20260807 22:38:12.287232 17803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:38:12.288847 17894 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:38:12.289517 17894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Bootstrap starting.
I20260807 22:38:12.293648 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 17803
I20260807 22:38:12.293947 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:43093
--local_ip_for_outbound_sockets=127.28.82.193
--tserver_master_addrs=127.28.82.254:46285
--webserver_port=35341
--webserver_interface=127.28.82.193
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:12.299017 17894 log.cc:824] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:12.301157 17894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:38:12.301461 17894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47: Bootstrap complete.
I20260807 22:38:12.303839 17894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:12.304064 17894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78cc1a7446664928847c112519abaa47, State: Initialized, Role: FOLLOWER
I20260807 22:38:12.304229 17894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:12.304307 17894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:38:12.304364 17894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:38:12.304414 17894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:38:12.305467 17894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:12.305558 17894 leader_election.cc:302] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78cc1a7446664928847c112519abaa47; no voters:
I20260807 22:38:12.305773 17894 leader_election.cc:288] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 22:38:12.305809 17899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:38:12.305920 17899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [term 3 LEADER]: Becoming Leader. State: Replica: 78cc1a7446664928847c112519abaa47, State: Running, Role: LEADER
I20260807 22:38:12.306018 17899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } }
I20260807 22:38:12.306053 17894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:12.306303 17899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78cc1a7446664928847c112519abaa47. Latest consensus state: current_term: 3 leader_uuid: "78cc1a7446664928847c112519abaa47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } } }
I20260807 22:38:12.306489 17899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:12.306305 17902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "78cc1a7446664928847c112519abaa47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cc1a7446664928847c112519abaa47" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 46285 } } }
I20260807 22:38:12.306787 17902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cc1a7446664928847c112519abaa47 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:12.306933 17910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:38:12.307717 17910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:38:12.307818 17910 catalog_manager.cc:1259] Loaded cluster ID: 60f71a10d2584db9a7eed4a2b3d24219
I20260807 22:38:12.307848 17910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 22:38:12.308198 17910 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8242b9c9ad google::LogMessage::Fail() at ??:0
@ 0x7f8242b9f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8242b9c4de google::LogMessage::Flush() at ??:0
@ 0x7f8242b9da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f824aabd0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f824aac68b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8244a6a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8244a5f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f82458c36db start_thread at ??:0
@ 0x7f824169a61f clone at ??:0
W20260807 22:38:12.392052 17897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:12.392282 17897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:12.392323 17897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:12.393443 17897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:12.393512 17897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:12.393558 17897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:38:12.395114 17897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:43093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=35341
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:12.395387 17897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:12.395634 17897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:12.401008 17951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:12.400995 17952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:12.401206 17897 server_base.cc:1034] running on GCE node
W20260807 22:38:12.401005 17955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:12.401548 17897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:12.401810 17897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:12.402976 17897 hybrid_clock.cc:648] HybridClock initialized: now 1786142292402960 us; error 31 us; skew 500 ppm
Aug 07 22:38:12 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142292, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:12.405738 17897 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:38:12.406189 17897 webserver.cc:460] Webserver started at http://127.28.82.193:35341/ using document root <none> and password file <none>
I20260807 22:38:12.406420 17897 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:12.406517 17897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:12.407998 17897 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:12.408774 17968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:12.409004 17897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:12.409108 17897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "d7233e3b3c694412a9b92680c588b74f"
format_stamp: "Formatted at 2026-08-07 22:38:10 on dist-test-slave-7kzw"
server_key: "bc88297a11cf2d1f9f37126937e7eb37"
server_key_iv: "99bc28b260669f3ead376efbc329cb33"
server_key_version: "encryptionkey@0"
I20260807 22:38:12.409447 17897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:12.459033 17897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:12.461527 17897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:12.461704 17897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:12.462014 17897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:12.462535 17980 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 22:38:12.464526 17897 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:38:12.464596 17897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 22:38:12.464648 17897 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:38:12.465328 17897 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:38:12.465390 17897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 22:38:12.465469 17980 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Bootstrap starting.
I20260807 22:38:12.473042 17897 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:43093
I20260807 22:38:12.473487 17897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:38:12.474061 18128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:43093 every 8 connection(s)
I20260807 22:38:12.477600 17980 log.cc:824] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:12.479893 17980 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:38:12.480201 17980 tablet_bootstrap.cc:492] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Bootstrap complete.
I20260807 22:38:12.481037 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 17897
I20260807 22:38:12.481317 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:39047
--local_ip_for_outbound_sockets=127.28.82.194
--tserver_master_addrs=127.28.82.254:46285
--webserver_port=34847
--webserver_interface=127.28.82.194
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:12.481467 17980 ts_tablet_manager.cc:1397] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260807 22:38:12.483311 17980 raft_consensus.cc:348] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:12.483579 17980 raft_consensus.cc:729] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7233e3b3c694412a9b92680c588b74f, State: Initialized, Role: FOLLOWER
I20260807 22:38:12.483717 17980 consensus_queue.cc:260] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:12.483983 17980 ts_tablet_manager.cc:1428] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260807 22:38:12.506918 18131 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:38:12.474279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:43481)
0807 22:38:12.474523 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.474578 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:12.475335 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:12.506626 (+ 31291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":109,"threads_started":1}
W20260807 22:38:12.507175 18129 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:46285 (0 consecutive failures): Network error: Failed to ping master at 127.28.82.254:46285: Client connection negotiation failed: client connection to 127.28.82.254:46285: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:38:12.507642 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.507474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:42405)
0807 22:38:12.507563 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.507616 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 22:38:12.508140 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.508038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:50641)
0807 22:38:12.508088 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.508116 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:38:12.508273 18129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:38:12.572680 18134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:12.572921 18134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:12.572962 18134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:12.574208 18134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:12.574277 18134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:12.574322 18134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:38:12.575855 18134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:39047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=34847
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:12.576138 18134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:12.576383 18134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:12.581645 18142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:12.581598 18141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:12.581776 18148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:12.581776 18134 server_base.cc:1034] running on GCE node
I20260807 22:38:12.582103 18134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:12.582355 18134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:12.583523 18134 hybrid_clock.cc:648] HybridClock initialized: now 1786142292583513 us; error 41 us; skew 500 ppm
Aug 07 22:38:12 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142292, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:12.586129 18134 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:38:12.586680 18134 webserver.cc:460] Webserver started at http://127.28.82.194:34847/ using document root <none> and password file <none>
I20260807 22:38:12.586928 18134 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:12.587000 18134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:12.588367 18134 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:12.589018 18159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:12.589206 18134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:12.589318 18134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "435ace9be1f84514ac8e01d7f6729ec1"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "bdf26c55bb16acca908285e443fe6e88"
server_key_iv: "2c1f00f25a6d3c9738ff95957404b6a0"
server_key_version: "encryptionkey@0"
I20260807 22:38:12.589661 18134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:12.607643 18134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:12.609545 18134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:12.609714 18134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:12.609951 18134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:12.610333 18134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:12.610390 18134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:12.610433 18134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:12.610479 18134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:12.616734 18134 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:39047
I20260807 22:38:12.616779 18322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:39047 every 8 connection(s)
I20260807 22:38:12.617118 18134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:38:12.618187 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.617814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:41649)
0807 22:38:12.617966 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.618045 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260807 22:38:12.618587 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 18134
W20260807 22:38:12.618721 18326 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:46285 (0 consecutive failures): Network error: Failed to ping master at 127.28.82.254:46285: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
I20260807 22:38:12.618860 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:32983
--local_ip_for_outbound_sockets=127.28.82.195
--tserver_master_addrs=127.28.82.254:46285
--webserver_port=38069
--webserver_interface=127.28.82.195
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:12.619120 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.618972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:38951)
0807 22:38:12.619077 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.619100 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 22:38:12.619490 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.619351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:40657)
0807 22:38:12.619426 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.619467 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 22:38:12.619673 18326 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:38:12.702941 18330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:12.703142 18330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:12.703166 18330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:12.704226 18330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:12.704263 18330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:12.704288 18330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:38:12.705655 18330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:32983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=38069
--tserver_master_addrs=127.28.82.254:46285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:12.705832 18330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:12.706003 18330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:12.711222 18344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:12.711244 18330 server_base.cc:1034] running on GCE node
W20260807 22:38:12.711196 18338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:12.711210 18339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:12.711661 18330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:12.711941 18330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:12.713114 18330 hybrid_clock.cc:648] HybridClock initialized: now 1786142292713096 us; error 55 us; skew 500 ppm
Aug 07 22:38:12 dist-test-slave-7kzw krb5kdc[16249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142292, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:12.715679 18330 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:38:12.716152 18330 webserver.cc:460] Webserver started at http://127.28.82.195:38069/ using document root <none> and password file <none>
I20260807 22:38:12.716400 18330 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:12.716475 18330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:12.717888 18330 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:12.718573 18355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:12.718763 18330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:12.718858 18330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "0a529469b3544087bb6952ac4424b2fd"
format_stamp: "Formatted at 2026-08-07 22:38:11 on dist-test-slave-7kzw"
server_key: "85eabdfd553f7b539595186049ff8993"
server_key_iv: "2247dce6da87c247b3bf7d863ed165da"
server_key_version: "encryptionkey@0"
I20260807 22:38:12.719230 18330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:12.732839 18330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:12.734827 18330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:12.734997 18330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:12.735247 18330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:12.735571 18330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:12.735630 18330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:12.735667 18330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:12.735715 18330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:12.742300 18330 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:32983
I20260807 22:38:12.742357 18516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:32983 every 8 connection(s)
I20260807 22:38:12.742718 18330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:38:12.743824 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.743366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:35967)
0807 22:38:12.743520 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.743656 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
W20260807 22:38:12.744306 18517 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:46285 (0 consecutive failures): Network error: Failed to ping master at 127.28.82.254:46285: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
I20260807 22:38:12.744626 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.744519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:56211)
0807 22:38:12.744572 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.744603 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 22:38:12.744956 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 18330
I20260807 22:38:12.745092 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:12.744968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:35763)
0807 22:38:12.745012 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:12.745059 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:38:12.745476 18517 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:38:13.509068 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:13.508818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:37817)
0807 22:38:13.508952 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:13.508996 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260807 22:38:13.620266 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:13.620037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:34531)
0807 22:38:13.620165 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:13.620202 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260807 22:38:13.701784 18525 raft_consensus.cc:482] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:13.701961 18525 raft_consensus.cc:504] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:13.702311 18525 leader_election.cc:288] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
Aug 07 22:38:13 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142292, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.194@KRBTEST.COM
Aug 07 22:38:13 dist-test-slave-7kzw krb5kdc[16249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142292, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.195@KRBTEST.COM
I20260807 22:38:13.714164 18529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:13.702882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:48723 (local address 127.28.82.195:32983)
0807 22:38:13.703013 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:13.703022 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:13.703040 (+ 18us) server_negotiation.cc:407] Connection header received
0807 22:38:13.703473 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:13.703475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:13.703524 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:13.703697 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:13.704459 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:13.705340 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:13.706362 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:13.706865 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:13.706952 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:13.710179 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:13.710182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:13.710182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:13.710195 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:13.712235 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:13.712831 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:13.712835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:13.712835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:13.712945 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:13.713252 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:13.713253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:13.713253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:13.713526 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:13.713631 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:13.713908 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 22:38:13.714054 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260807 22:38:13.714205 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:13.702694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:38167 (local address 127.28.82.194:39047)
0807 22:38:13.702910 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:13.702925 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:13.702965 (+ 40us) server_negotiation.cc:407] Connection header received
0807 22:38:13.703460 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:13.703463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:13.703507 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:13.703623 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:13.704435 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:13.705330 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:13.706421 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:13.706938 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:13.707069 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:13.709641 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:13.709650 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:13.709650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:13.709662 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:13.712106 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:13.712596 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:13.712600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:13.712600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:13.712724 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:13.713082 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:13.713083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:13.713084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:13.713499 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:13.713577 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:13.713900 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 22:38:13.714054 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260807 22:38:13.714200 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:13.702557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:39047 (local address 127.28.82.193:38167)
0807 22:38:13.702613 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:13.702631 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:13.702696 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:13.703704 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:13.703707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:13.704120 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:13.704304 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:13.704309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:13.705445 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:13.705446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:13.706254 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:13.706257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:13.707039 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:13.707040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:13.707094 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:13.707809 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:13.707820 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:13.709491 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:13.712209 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:13.712220 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:13.712222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:13.712515 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:13.712822 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:13.712823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:13.712823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:13.712964 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:13.713583 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:13.713591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:13.713737 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 22:38:13.713835 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 22:38:13.714065 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":90}
I20260807 22:38:13.714735 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:13.702750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:32983 (local address 127.28.82.193:48723)
0807 22:38:13.702892 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:13.702903 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:13.702952 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:13.703742 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:13.703743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:13.704120 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:13.704304 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:13.704309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:13.705455 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:13.705456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:13.706252 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:13.706254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:13.706965 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:13.706966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:13.707086 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:13.707811 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:13.707820 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:13.710020 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:13.712384 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:13.712385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:13.712386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:13.712709 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:13.713098 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:13.713099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:13.713100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:13.713170 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:13.713626 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:13.713627 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:13.713728 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 22:38:13.713834 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 22:38:13.714154 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":149,"thread_start_us":71,"threads_started":1}
W20260807 22:38:13.715564 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
W20260807 22:38:13.715677 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
I20260807 22:38:13.715711 17972 leader_election.cc:302] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7233e3b3c694412a9b92680c588b74f; no voters: 0a529469b3544087bb6952ac4424b2fd, 435ace9be1f84514ac8e01d7f6729ec1
I20260807 22:38:13.715824 18525 raft_consensus.cc:2726] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:38:13.746171 18530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:13.745903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:35383)
0807 22:38:13.746069 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:13.746129 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260807 22:38:14.509886 18532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:14.509649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:38237)
0807 22:38:14.509797 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:14.509836 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260807 22:38:14.621168 18533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:14.620901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:34643)
0807 22:38:14.621052 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:14.621101 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260807 22:38:14.747087 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:14.746838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:44055)
0807 22:38:14.746993 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:14.747037 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260807 22:38:15.321661 18535 raft_consensus.cc:482] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:15.321811 18535 raft_consensus.cc:504] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:15.321993 18535 leader_election.cc:288] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
W20260807 22:38:15.322564 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
W20260807 22:38:15.322731 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
I20260807 22:38:15.322770 17972 leader_election.cc:302] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7233e3b3c694412a9b92680c588b74f; no voters: 0a529469b3544087bb6952ac4424b2fd, 435ace9be1f84514ac8e01d7f6729ec1
I20260807 22:38:15.322867 18535 raft_consensus.cc:2726] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:38:15.510977 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:15.510693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:37989)
0807 22:38:15.510842 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:15.510911 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260807 22:38:15.621943 18537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:15.621707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:46683)
0807 22:38:15.621845 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:15.621884 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260807 22:38:15.747829 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:15.747620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:36885)
0807 22:38:15.747749 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:15.747787 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260807 22:38:16.511785 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:16.511420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:55371)
0807 22:38:16.511554 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:16.511600 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260807 22:38:16.624212 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:16.623967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:40773)
0807 22:38:16.624113 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:16.624162 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
I20260807 22:38:16.748584 18544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:16.748320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:55397)
0807 22:38:16.748493 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:16.748536 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":64,"threads_started":1}
I20260807 22:38:17.512725 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:17.512508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:54471)
0807 22:38:17.512631 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:17.512678 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260807 22:38:17.531152 18547 raft_consensus.cc:482] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:17.531376 18547 raft_consensus.cc:504] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:17.531581 18547 leader_election.cc:288] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
W20260807 22:38:17.532079 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
W20260807 22:38:17.532166 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
I20260807 22:38:17.532183 17972 leader_election.cc:302] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7233e3b3c694412a9b92680c588b74f; no voters: 0a529469b3544087bb6952ac4424b2fd, 435ace9be1f84514ac8e01d7f6729ec1
I20260807 22:38:17.532290 18547 raft_consensus.cc:2726] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:38:17.625063 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:17.624842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:48167)
0807 22:38:17.624977 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:17.625016 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260807 22:38:17.749471 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:17.749185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:44771)
0807 22:38:17.749360 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:17.749425 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260807 22:38:18.513571 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:18.513295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:57501)
0807 22:38:18.513466 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:18.513519 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260807 22:38:18.625818 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:18.625565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:53935)
0807 22:38:18.625722 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:18.625765 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260807 22:38:18.750313 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:18.750036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:33891)
0807 22:38:18.750220 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:18.750262 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":83,"threads_started":1}
I20260807 22:38:19.514283 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:19.514059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:59187)
0807 22:38:19.514194 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:19.514233 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260807 22:38:19.626675 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:19.626398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:46883)
0807 22:38:19.626578 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:19.626623 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":145,"threads_started":1}
I20260807 22:38:19.747177 18559 raft_consensus.cc:482] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:19.747324 18559 raft_consensus.cc:504] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:19.747532 18559 leader_election.cc:288] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
W20260807 22:38:19.748014 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
W20260807 22:38:19.748097 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
I20260807 22:38:19.748116 17972 leader_election.cc:302] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7233e3b3c694412a9b92680c588b74f; no voters: 0a529469b3544087bb6952ac4424b2fd, 435ace9be1f84514ac8e01d7f6729ec1
I20260807 22:38:19.748226 18559 raft_consensus.cc:2726] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:38:19.751055 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:19.750826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:56669)
0807 22:38:19.750959 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:19.750997 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260807 22:38:20.515105 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:20.514835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:38765)
0807 22:38:20.514982 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:20.515029 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260807 22:38:20.627460 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:20.627216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:37315)
0807 22:38:20.627368 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:20.627410 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260807 22:38:20.751819 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:20.751552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:38683)
0807 22:38:20.751707 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:20.751746 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260807 22:38:21.515878 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:21.515635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:52085)
0807 22:38:21.515779 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:21.515827 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260807 22:38:21.628180 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:21.627931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:57597)
0807 22:38:21.628081 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:21.628127 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260807 22:38:21.752599 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:21.752369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:46325)
0807 22:38:21.752514 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:21.752553 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260807 22:38:22.516721 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:22.516410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:47083)
0807 22:38:22.516585 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:22.516671 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260807 22:38:22.628973 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:22.628703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:53673)
0807 22:38:22.628881 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:22.628923 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260807 22:38:22.753371 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:22.753117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:59667)
0807 22:38:22.753275 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:22.753314 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260807 22:38:23.517616 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:23.517284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:53019)
0807 22:38:23.517511 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:23.517561 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":115,"threads_started":1}
I20260807 22:38:23.629946 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:23.629667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:35209)
0807 22:38:23.629844 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:23.629892 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
I20260807 22:38:23.754297 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:23.754038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:51619)
0807 22:38:23.754203 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:23.754245 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260807 22:38:24.518386 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:24.518154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:35765)
0807 22:38:24.518288 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:24.518335 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260807 22:38:24.630892 18577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:24.630612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:48881)
0807 22:38:24.630790 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:24.630833 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260807 22:38:24.755023 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:24.754790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:55927)
0807 22:38:24.754934 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:24.754973 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260807 22:38:25.519309 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:25.519054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:33549)
0807 22:38:25.519217 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:25.519260 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260807 22:38:25.631671 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:25.631418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:48081)
0807 22:38:25.631573 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:25.631613 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260807 22:38:25.755788 18584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:25.755492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:54989)
0807 22:38:25.755698 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:25.755739 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":99,"threads_started":1}
I20260807 22:38:26.520079 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:26.519845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:35701)
0807 22:38:26.519983 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:26.520028 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260807 22:38:26.585510 18586 raft_consensus.cc:482] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:26.585624 18586 raft_consensus.cc:504] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435ace9be1f84514ac8e01d7f6729ec1" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 39047 } } peers { permanent_uuid: "0a529469b3544087bb6952ac4424b2fd" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 32983 } } peers { permanent_uuid: "d7233e3b3c694412a9b92680c588b74f" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43093 } }
I20260807 22:38:26.585844 18586 leader_election.cc:288] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047), 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983)
W20260807 22:38:26.586351 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 435ace9be1f84514ac8e01d7f6729ec1 (127.28.82.194:39047): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
W20260807 22:38:26.586459 17972 leader_election.cc:341] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a529469b3544087bb6952ac4424b2fd (127.28.82.195:32983): Not found: Tablet not found: b2b44c35bf7c46848e215b983d07f334
I20260807 22:38:26.586520 17972 leader_election.cc:302] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7233e3b3c694412a9b92680c588b74f; no voters: 0a529469b3544087bb6952ac4424b2fd, 435ace9be1f84514ac8e01d7f6729ec1
I20260807 22:38:26.586665 18586 raft_consensus.cc:2726] T b2b44c35bf7c46848e215b983d07f334 P d7233e3b3c694412a9b92680c588b74f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:38:26.632551 18587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:26.632307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:43685)
0807 22:38:26.632461 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:26.632503 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260807 22:38:26.756642 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:26.756366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:60731)
0807 22:38:26.756512 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:26.756592 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260807 22:38:27.520859 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:27.520638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.193:46059)
0807 22:38:27.520774 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:27.520811 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260807 22:38:27.633466 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:27.633237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.194:58753)
0807 22:38:27.633378 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:27.633417 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260807 22:38:27.748775 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 17897
I20260807 22:38:27.759443 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:27.759230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:46285 (local address 127.28.82.195:52127)
0807 22:38:27.759356 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:27.759398 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:46285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260807 22:38:27.764150 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 18134
I20260807 22:38:27.772701 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 18330
2026-08-07T22:38:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18548 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:38:27 dist-test-slave-7kzw krb5kdc[18598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:38:27 dist-test-slave-7kzw krb5kdc[18598](info): set up 2 sockets
Aug 07 22:38:27 dist-test-slave-7kzw krb5kdc[18598](info): commencing operation
krb5kdc: starting...
W20260807 22:38:28.993925 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:38:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:38:29.115447 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:35113
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:29.198269 18616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:29.198524 18616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:29.198572 18616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:29.199679 18616 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:38:29.199741 18616 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:38:29.199762 18616 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:38:29.199781 18616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:29.199808 18616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:29.199841 18616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:29.199869 18616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:29.199896 18616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:29.201339 18616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:35113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:29.201619 18616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:29.201843 18616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:29.207039 18621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:29.207149 18616 server_base.cc:1034] running on GCE node
W20260807 22:38:29.207221 18629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:29.207017 18623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:29.207656 18616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:29.207986 18616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:29.209146 18616 hybrid_clock.cc:648] HybridClock initialized: now 1786142309209135 us; error 42 us; skew 500 ppm
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:29.211808 18616 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:38:29.212221 18616 webserver.cc:460] Webserver started at http://127.28.82.254:40687/ using document root <none> and password file <none>
I20260807 22:38:29.212435 18616 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:29.212478 18616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:29.212620 18616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:29.213632 18616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "e293f60604274fda9207059b5538cbe9"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "8dd4a1dec8673396b3396fc8a5eea0d5"
server_key_iv: "c24bac387d8bcecff9f7351b3d2dfb79"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.214000 18616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "e293f60604274fda9207059b5538cbe9"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "8dd4a1dec8673396b3396fc8a5eea0d5"
server_key_iv: "c24bac387d8bcecff9f7351b3d2dfb79"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.215411 18616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:38:29.216189 18641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.216363 18616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:29.216470 18616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "e293f60604274fda9207059b5538cbe9"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "8dd4a1dec8673396b3396fc8a5eea0d5"
server_key_iv: "c24bac387d8bcecff9f7351b3d2dfb79"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.216542 18616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:29.228477 18616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:29.230351 18616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:29.230549 18616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:29.234368 18616 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:35113
I20260807 22:38:29.234436 18713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:35113 every 8 connection(s)
I20260807 22:38:29.234845 18616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:38:29.235682 18714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:29.237942 18714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Bootstrap starting.
I20260807 22:38:29.238612 18714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:29.238919 18714 log.cc:824] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:29.239600 18714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: No bootstrap required, opened a new log
I20260807 22:38:29.240530 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 18616
I20260807 22:38:29.240621 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:38:29.240821 29003 external_mini_cluster.cc:1428] Setting key a7fe8bf4e24d19bc991345e28fc48aff
I20260807 22:38:29.241533 18714 raft_consensus.cc:348] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:38:29.241696 18714 raft_consensus.cc:374] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:29.241746 18714 raft_consensus.cc:729] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e293f60604274fda9207059b5538cbe9, State: Initialized, Role: FOLLOWER
I20260807 22:38:29.241930 18714 consensus_queue.cc:260] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:38:29.242030 18714 raft_consensus.cc:388] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:38:29.242066 18714 raft_consensus.cc:482] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:38:29.242117 18714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:29.243126 18714 raft_consensus.cc:504] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:38:29.243228 18714 leader_election.cc:302] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e293f60604274fda9207059b5538cbe9; no voters:
I20260807 22:38:29.243414 18714 leader_election.cc:288] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:38:29.243453 18719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:38:29.243616 18719 raft_consensus.cc:686] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 1 LEADER]: Becoming Leader. State: Replica: e293f60604274fda9207059b5538cbe9, State: Running, Role: LEADER
I20260807 22:38:29.243685 18714 sys_catalog.cc:564] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:29.244886 18719 consensus_queue.cc:237] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:38:29.245528 18720 sys_catalog.cc:455] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e293f60604274fda9207059b5538cbe9. Latest consensus state: current_term: 1 leader_uuid: "e293f60604274fda9207059b5538cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } } }
I20260807 22:38:29.245532 18721 sys_catalog.cc:455] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e293f60604274fda9207059b5538cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } } }
I20260807 22:38:29.245659 18720 sys_catalog.cc:458] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:29.245954 18743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:38:29.245672 18721 sys_catalog.cc:458] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: This master's current role is: LEADER
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:29.246492 18743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:38:29.250018 18743 catalog_manager.cc:1347] Generated new cluster ID: 62e893366b4649309dcbb5b9fea9ca71
I20260807 22:38:29.250074 18743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:38:29.252148 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.241333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.28.82.254:35113)
0807 22:38:29.241511 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.241519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.241543 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:38:29.241631 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.241636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.241683 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.241830 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:29.242630 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.243773 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.244961 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.245591 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.245742 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.248444 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:29.248447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:29.248447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:29.248470 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:29.250539 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.250908 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.250911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.250911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.251019 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.251223 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.251224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.251225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.251460 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:29.251610 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.251723 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.251872 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260807 22:38:29.263717 18743 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:38:29.264413 18743 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:38:29.269929 18743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Generated new TSK 0
I20260807 22:38:29.270097 18743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:38:29.303750 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:35113
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:29.398442 18753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:29.398697 18753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:29.398737 18753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:29.399914 18753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:29.399958 18753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:29.399984 18753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:38:29.401404 18753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:29.401643 18753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:29.401881 18753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:29.406987 18766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:29.407101 18761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:29.407133 18763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:29.407212 18753 server_base.cc:1034] running on GCE node
I20260807 22:38:29.407537 18753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:29.407809 18753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:29.408964 18753 hybrid_clock.cc:648] HybridClock initialized: now 1786142309408953 us; error 40 us; skew 500 ppm
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:29.411608 18753 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:38:29.412068 18753 webserver.cc:460] Webserver started at http://127.28.82.193:45439/ using document root <none> and password file <none>
I20260807 22:38:29.412293 18753 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:29.412359 18753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:29.412501 18753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:29.413551 18753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "0de04c931e9a48fba59cd81ddd8478e3"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "abc4696c3bad19436d3cd61b2218a081"
server_key_iv: "62a46e460de126d0abd22a5a06cb8e6e"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.413913 18753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "0de04c931e9a48fba59cd81ddd8478e3"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "abc4696c3bad19436d3cd61b2218a081"
server_key_iv: "62a46e460de126d0abd22a5a06cb8e6e"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.415336 18753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:38:29.416134 18776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.416301 18753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.416406 18753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "0de04c931e9a48fba59cd81ddd8478e3"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "abc4696c3bad19436d3cd61b2218a081"
server_key_iv: "62a46e460de126d0abd22a5a06cb8e6e"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.416477 18753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:29.431692 18753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:29.433512 18753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:29.433677 18753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:29.433911 18753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:29.434239 18753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:29.434302 18753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.434345 18753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:29.434376 18753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.440569 18753 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:45721
I20260807 22:38:29.440680 18952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:45721 every 8 connection(s)
I20260807 22:38:29.440987 18753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:29.450713 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 18753
I20260807 22:38:29.450855 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:38:29.451081 29003 external_mini_cluster.cc:1428] Setting key 81ee4346118733694716fc3108328aab
I20260807 22:38:29.452924 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.441826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:37215 (local address 127.28.82.254:35113)
0807 22:38:29.441873 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.441875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.442101 (+ 226us) server_negotiation.cc:407] Connection header received
0807 22:38:29.442714 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.442715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.442732 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.442785 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:29.443822 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.444510 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.445995 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.446353 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.446445 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.448709 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:29.448711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:29.448712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:29.448722 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:29.450877 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.451569 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.451570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.451571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.451647 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.451975 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.451976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.451977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.452149 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:29.452229 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.452651 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.452768 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 22:38:29.454044 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.441794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:37215)
0807 22:38:29.441982 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.442017 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:29.442602 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:29.442856 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:29.442860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:29.443443 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:29.443714 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.443720 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.444614 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.444615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.445889 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.445893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.446452 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.446453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.446634 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.447160 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:29.447168 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:29.448594 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:29.451025 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:29.451033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:29.451036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:29.451436 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:29.451742 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:29.451744 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:29.451744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:29.451910 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:29.452207 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:29.452216 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:29.452429 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 22:38:29.453456 (+ 1027us) client_negotiation.cc:240] Negotiation successful
0807 22:38:29.453702 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260807 22:38:29.456977 18953 heartbeater.cc:347] Connected to a master server at 127.28.82.254:35113
I20260807 22:38:29.457087 18953 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:29.457286 18953 heartbeater.cc:511] Master 127.28.82.254:35113 requested a full tablet report, sending...
I20260807 22:38:29.457866 18671 ts_manager.cc:194] Registered new tserver with Master: 0de04c931e9a48fba59cd81ddd8478e3 (127.28.82.193:45721)
I20260807 22:38:29.458822 18671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:37215
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:38:29.485692 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:35113
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:29.572757 18965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:29.572950 18965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:29.572973 18965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:29.574024 18965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:29.574055 18965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:29.574081 18965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:38:29.575428 18965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:29.575608 18965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:29.575776 18965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:29.580929 18974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:29.580947 18976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:29.580938 18973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:29.581033 18965 server_base.cc:1034] running on GCE node
I20260807 22:38:29.581380 18965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:29.581622 18965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:29.582779 18965 hybrid_clock.cc:648] HybridClock initialized: now 1786142309582775 us; error 60 us; skew 500 ppm
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:29.585266 18965 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:38:29.585677 18965 webserver.cc:460] Webserver started at http://127.28.82.194:43313/ using document root <none> and password file <none>
I20260807 22:38:29.585896 18965 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:29.585961 18965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:29.586108 18965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:29.587216 18965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "97cf2f6f50784e1caf5dc8e96509adb7"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "c3faf3c608022bb6a1b3408c1420edcf"
server_key_iv: "9b6ea954af49521b2aa572ef0f7db370"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.587589 18965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "97cf2f6f50784e1caf5dc8e96509adb7"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "c3faf3c608022bb6a1b3408c1420edcf"
server_key_iv: "9b6ea954af49521b2aa572ef0f7db370"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.588965 18965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:38:29.589694 18984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.589879 18965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:38:29.589982 18965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "97cf2f6f50784e1caf5dc8e96509adb7"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "c3faf3c608022bb6a1b3408c1420edcf"
server_key_iv: "9b6ea954af49521b2aa572ef0f7db370"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.590054 18965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:29.605547 18965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:29.607415 18965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:29.607571 18965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:29.607797 18965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:29.608096 18965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:29.608148 18965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.608191 18965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:29.608242 18965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.614248 18965 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:34317
I20260807 22:38:29.614296 19147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:34317 every 8 connection(s)
I20260807 22:38:29.614645 18965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:29.621937 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 18965
I20260807 22:38:29.622052 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:38:29.622277 29003 external_mini_cluster.cc:1428] Setting key e9d0d9ec2228019c8b996aa63e0ac7e5
I20260807 22:38:29.624222 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.615275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:41075)
0807 22:38:29.615436 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.615477 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:29.616066 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:29.616301 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:29.616304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:29.616872 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:29.617054 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.617058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.617729 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.617730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.618650 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.618658 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.618980 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.618981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.619079 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.619623 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:29.619632 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:29.621112 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:29.622913 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:29.622922 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:29.622925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:29.623181 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:29.623415 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:29.623416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:29.623416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:29.623509 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:29.623768 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:29.623771 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:29.623871 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 22:38:29.623944 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 22:38:29.624060 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260807 22:38:29.624214 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.615321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:41075 (local address 127.28.82.254:35113)
0807 22:38:29.615387 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.615389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.615560 (+ 171us) server_negotiation.cc:407] Connection header received
0807 22:38:29.616155 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.616156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.616167 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.616243 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:29.617135 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.617648 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.618724 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.618909 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.618966 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.621194 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:29.621209 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:29.621210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:29.621217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:29.622811 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.623283 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.623285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.623285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.623350 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.623563 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.623564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.623564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.623708 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:29.623783 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.623987 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.624079 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:38:29.624692 19149 heartbeater.cc:347] Connected to a master server at 127.28.82.254:35113
I20260807 22:38:29.624799 19149 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:29.625005 19149 heartbeater.cc:511] Master 127.28.82.254:35113 requested a full tablet report, sending...
I20260807 22:38:29.625423 18671 ts_manager.cc:194] Registered new tserver with Master: 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
I20260807 22:38:29.625798 18671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:41075
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:38:29.649788 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:35113
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:29.736109 19154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:29.736317 19154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:29.736341 19154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:29.737424 19154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:29.737463 19154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:29.737488 19154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:38:29.738916 19154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:29.739097 19154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:29.739285 19154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:29.744434 19161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:29.744432 19164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:29.744558 19154 server_base.cc:1034] running on GCE node
W20260807 22:38:29.744586 19162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:29.744968 19154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:29.745260 19154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:29.746420 19154 hybrid_clock.cc:648] HybridClock initialized: now 1786142309746406 us; error 29 us; skew 500 ppm
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:29.748912 19154 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:38:29.749392 19154 webserver.cc:460] Webserver started at http://127.28.82.195:45173/ using document root <none> and password file <none>
I20260807 22:38:29.749650 19154 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:29.749716 19154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:29.749867 19154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:38:29.750985 19154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "08c64815edec4d6baf964973ee1dd0fc"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "23cffd11a33667a01e7826de616acd61"
server_key_iv: "6fd871eedb31629eb2acd1155568da1b"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.751351 19154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "08c64815edec4d6baf964973ee1dd0fc"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "23cffd11a33667a01e7826de616acd61"
server_key_iv: "6fd871eedb31629eb2acd1155568da1b"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.752753 19154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:29.753520 19173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.753809 19154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:29.753911 19154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "08c64815edec4d6baf964973ee1dd0fc"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "23cffd11a33667a01e7826de616acd61"
server_key_iv: "6fd871eedb31629eb2acd1155568da1b"
server_key_version: "encryptionkey@0"
I20260807 22:38:29.753981 19154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:29.762743 19154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:29.764729 19154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:29.764878 19154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:29.765098 19154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:29.765415 19154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:29.765468 19154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.765509 19154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:29.765537 19154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:29.771842 19154 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:43617
I20260807 22:38:29.771890 19344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:43617 every 8 connection(s)
I20260807 22:38:29.772199 19154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:38:29.774904 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 19154
I20260807 22:38:29.774981 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:38:29.775158 29003 external_mini_cluster.cc:1428] Setting key 09e5d73b891c4d8a34520cf44b40e74b
Aug 07 22:38:29 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142309, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:29.781760 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.772921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:36171 (local address 127.28.82.254:35113)
0807 22:38:29.772984 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.772986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.773198 (+ 212us) server_negotiation.cc:407] Connection header received
0807 22:38:29.773737 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.773739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.773753 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.773812 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:29.774820 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.775515 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.776502 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.776725 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.776811 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.779003 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:29.779005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:29.779005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:29.779014 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:29.780427 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.780857 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.780858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.780858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.780913 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.781157 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.781158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.781158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.781266 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:29.781321 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.781623 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.781687 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:38:29.781984 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.772903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:36171)
0807 22:38:29.773079 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.773120 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:29.773624 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:29.773870 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:29.773874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:29.774512 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:29.774715 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.774720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.775611 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.775612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.776409 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.776412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.776825 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.776827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.776949 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.777461 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:29.777469 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:29.778906 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:29.780519 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:29.780527 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:29.780529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:29.780790 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:29.780982 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:29.780983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:29.780983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:29.781100 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:29.781353 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:29.781360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:29.781491 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 22:38:29.781599 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 22:38:29.781740 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260807 22:38:29.782599 19345 heartbeater.cc:347] Connected to a master server at 127.28.82.254:35113
I20260807 22:38:29.782738 19345 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:29.782930 19345 heartbeater.cc:511] Master 127.28.82.254:35113 requested a full tablet report, sending...
I20260807 22:38:29.783352 18671 ts_manager.cc:194] Registered new tserver with Master: 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617)
I20260807 22:38:29.783689 18671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:36171
I20260807 22:38:29.786823 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:38:29.793938 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.788060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54912 (local address 127.28.82.254:35113)
0807 22:38:29.788101 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.788102 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.788217 (+ 115us) server_negotiation.cc:407] Connection header received
0807 22:38:29.788267 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.788268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.788276 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.788324 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:29.788872 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.789320 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.790034 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.790202 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.790258 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.791456 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:29.791458 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:29.791458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:29.791464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:29.792909 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.793228 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.793229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.793229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.793290 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:29.793450 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:29.793450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:29.793450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:29.793596 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:29.793695 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.793811 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.793879 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 22:38:29.795495 18671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:38:29.797024 18671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:38:29.804204 19359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.799436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:45721 (local address 127.0.0.1:52300)
0807 22:38:29.799779 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.799822 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:29.799874 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:29.800318 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:29.800321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:29.800324 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:29.800387 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.800389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.801734 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.801734 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.803081 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.803084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.803872 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.803874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.803969 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.803983 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:38:29.804069 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 22:38:29.804125 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":55,"thread_start_us":71,"threads_started":1}
I20260807 22:38:29.804214 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.799554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.28.82.193:45721)
0807 22:38:29.800133 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.800136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.800145 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:38:29.800179 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.800180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.800193 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.800305 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:29.800467 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.801648 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.803183 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.803778 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.803860 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.803923 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.804085 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.804129 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":126,"threads_started":1}
I20260807 22:38:29.804494 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.799649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.28.82.194:34317)
0807 22:38:29.800053 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.800056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.800481 (+ 425us) server_negotiation.cc:407] Connection header received
0807 22:38:29.800763 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.800766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.800782 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.800833 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:29.801042 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.801989 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.803087 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.803812 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.803883 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.803935 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.804151 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.804197 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":90,"threads_started":1}
I20260807 22:38:29.804786 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.799672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:43617 (local address 127.0.0.1:40834)
0807 22:38:29.800530 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.800539 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:29.800601 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:29.800779 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:29.800780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:29.800782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:29.800830 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.800831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.801648 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.801649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.803082 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.803084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.804178 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.804179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.804231 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.804235 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:38:29.804282 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 22:38:29.804668 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":56,"thread_start_us":83,"threads_started":1}
I20260807 22:38:29.804870 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.799481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:34317 (local address 127.0.0.1:49858)
0807 22:38:29.800402 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.800410 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:29.800668 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:29.800930 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:29.800930 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:29.800932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:29.800973 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.800974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.802112 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.802112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.802998 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.803000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.803972 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.803973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.804038 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.804042 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:38:29.804706 (+ 664us) client_negotiation.cc:240] Negotiation successful
0807 22:38:29.804747 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"thread_start_us":81,"threads_started":1}
I20260807 22:38:29.805049 19366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.799841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40834 (local address 127.28.82.195:43617)
0807 22:38:29.799997 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.800002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.800598 (+ 596us) server_negotiation.cc:407] Connection header received
0807 22:38:29.800670 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.800672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.800684 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.800760 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:29.800883 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.801546 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.803170 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.804016 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.804082 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.804145 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.804482 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.804533 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260807 22:38:29.804970 18845 tablet_service.cc:1467] Processing CreateTablet for tablet f7ae2a22d5ff485db3b94963a694ca76 (DEFAULT_TABLE table=test-table [id=81759f536c274e2a9e89d384f9ff1879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:29.805344 18845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ae2a22d5ff485db3b94963a694ca76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:29.805606 19243 tablet_service.cc:1467] Processing CreateTablet for tablet f7ae2a22d5ff485db3b94963a694ca76 (DEFAULT_TABLE table=test-table [id=81759f536c274e2a9e89d384f9ff1879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:29.805895 19243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ae2a22d5ff485db3b94963a694ca76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:29.806915 19057 tablet_service.cc:1467] Processing CreateTablet for tablet f7ae2a22d5ff485db3b94963a694ca76 (DEFAULT_TABLE table=test-table [id=81759f536c274e2a9e89d384f9ff1879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:29.807256 19057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ae2a22d5ff485db3b94963a694ca76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:29.808274 19367 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Bootstrap starting.
I20260807 22:38:29.808944 19367 tablet_bootstrap.cc:654] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:29.809458 19367 log.cc:824] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:29.809533 19369 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: Bootstrap starting.
I20260807 22:38:29.810155 19367 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: No bootstrap required, opened a new log
I20260807 22:38:29.810236 19367 ts_tablet_manager.cc:1397] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:38:29.810271 19369 tablet_bootstrap.cc:654] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:29.810796 19371 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Bootstrap starting.
I20260807 22:38:29.810912 19369 log.cc:824] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:29.811587 19369 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: No bootstrap required, opened a new log
I20260807 22:38:29.811583 19371 tablet_bootstrap.cc:654] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:29.811643 19369 ts_tablet_manager.cc:1397] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:29.812281 19371 log.cc:824] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:29.812456 19367 raft_consensus.cc:348] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.812649 19367 raft_consensus.cc:374] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:29.812695 19367 raft_consensus.cc:729] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de04c931e9a48fba59cd81ddd8478e3, State: Initialized, Role: FOLLOWER
I20260807 22:38:29.812839 19367 consensus_queue.cc:260] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.813143 19367 ts_tablet_manager.cc:1428] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:38:29.813278 19371 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: No bootstrap required, opened a new log
I20260807 22:38:29.813303 18953 heartbeater.cc:503] Master 127.28.82.254:35113 was elected leader, sending a full tablet report...
I20260807 22:38:29.813350 19371 ts_tablet_manager.cc:1397] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 22:38:29.813241 19369 raft_consensus.cc:348] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.813406 19369 raft_consensus.cc:374] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:29.813459 19369 raft_consensus.cc:729] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c64815edec4d6baf964973ee1dd0fc, State: Initialized, Role: FOLLOWER
I20260807 22:38:29.813570 19369 consensus_queue.cc:260] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.813843 19369 ts_tablet_manager.cc:1428] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:29.813885 19345 heartbeater.cc:503] Master 127.28.82.254:35113 was elected leader, sending a full tablet report...
I20260807 22:38:29.815165 19371 raft_consensus.cc:348] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.815294 19371 raft_consensus.cc:374] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:29.815316 19371 raft_consensus.cc:729] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97cf2f6f50784e1caf5dc8e96509adb7, State: Initialized, Role: FOLLOWER
I20260807 22:38:29.815446 19371 consensus_queue.cc:260] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.815735 19371 ts_tablet_manager.cc:1428] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 22:38:29.815773 19149 heartbeater.cc:503] Master 127.28.82.254:35113 was elected leader, sending a full tablet report...
I20260807 22:38:29.858552 19377 raft_consensus.cc:482] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:29.858690 19377 raft_consensus.cc:504] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.859076 19377 leader_election.cc:288] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317), 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617)
I20260807 22:38:29.861969 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.859464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:36889 (local address 127.28.82.194:34317)
0807 22:38:29.859528 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.859530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.859538 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:38:29.859578 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.859579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.859588 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.859639 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:29.859793 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.860271 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.861213 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.861621 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.861705 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.861714 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.861870 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.861907 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:38:29.861970 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.859255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:34317 (local address 127.28.82.193:36889)
0807 22:38:29.859312 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.859325 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:29.859369 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:29.859647 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:29.859649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:29.859650 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:29.859724 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.859725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.860351 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.860352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.861119 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.861121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.861703 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.861704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.861788 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.861791 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:38:29.861850 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:38:29.861894 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 22:38:29.862114 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.859402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:43617 (local address 127.28.82.193:48129)
0807 22:38:29.859591 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.859600 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:29.859687 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:29.859854 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:29.859855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:29.859856 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:29.859933 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.859934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.860541 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.860542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.861293 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:29.861295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.861850 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.861852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:29.861923 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.861927 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:38:29.861993 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:38:29.862035 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":122,"threads_started":1}
I20260807 22:38:29.862112 19366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.859459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:48129 (local address 127.28.82.195:43617)
0807 22:38:29.859511 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.859513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.859668 (+ 155us) server_negotiation.cc:407] Connection header received
0807 22:38:29.859774 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.859775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.859783 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.859836 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:29.860005 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.860462 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.861387 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.861768 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.861849 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.861857 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.862012 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.862051 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 22:38:29.862354 19086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ae2a22d5ff485db3b94963a694ca76" candidate_uuid: "0de04c931e9a48fba59cd81ddd8478e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" is_pre_election: true
I20260807 22:38:29.862541 19086 raft_consensus.cc:2454] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0de04c931e9a48fba59cd81ddd8478e3 in term 0.
I20260807 22:38:29.862761 18778 leader_election.cc:302] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0de04c931e9a48fba59cd81ddd8478e3, 97cf2f6f50784e1caf5dc8e96509adb7; no voters:
I20260807 22:38:29.862917 19377 raft_consensus.cc:2781] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:38:29.862994 19377 raft_consensus.cc:482] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:38:29.863025 19377 raft_consensus.cc:3037] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:29.863113 19276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ae2a22d5ff485db3b94963a694ca76" candidate_uuid: "0de04c931e9a48fba59cd81ddd8478e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c64815edec4d6baf964973ee1dd0fc" is_pre_election: true
I20260807 22:38:29.863257 19276 raft_consensus.cc:2454] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0de04c931e9a48fba59cd81ddd8478e3 in term 0.
I20260807 22:38:29.863847 19377 raft_consensus.cc:504] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.863977 19377 leader_election.cc:288] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 1 election: Requested vote from peers 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317), 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617)
I20260807 22:38:29.864120 19086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ae2a22d5ff485db3b94963a694ca76" candidate_uuid: "0de04c931e9a48fba59cd81ddd8478e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97cf2f6f50784e1caf5dc8e96509adb7"
I20260807 22:38:29.864193 19086 raft_consensus.cc:3037] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:29.864172 19276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ae2a22d5ff485db3b94963a694ca76" candidate_uuid: "0de04c931e9a48fba59cd81ddd8478e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c64815edec4d6baf964973ee1dd0fc"
I20260807 22:38:29.864248 19276 raft_consensus.cc:3037] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:29.864920 19086 raft_consensus.cc:2454] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0de04c931e9a48fba59cd81ddd8478e3 in term 1.
I20260807 22:38:29.865020 19276 raft_consensus.cc:2454] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0de04c931e9a48fba59cd81ddd8478e3 in term 1.
W20260807 22:38:29.865139 19150 tablet.cc:2369] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:38:29.865195 18778 leader_election.cc:302] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c64815edec4d6baf964973ee1dd0fc, 0de04c931e9a48fba59cd81ddd8478e3; no voters:
I20260807 22:38:29.865317 19377 raft_consensus.cc:2781] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:38:29.865492 19377 raft_consensus.cc:686] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 LEADER]: Becoming Leader. State: Replica: 0de04c931e9a48fba59cd81ddd8478e3, State: Running, Role: LEADER
I20260807 22:38:29.865597 19377 consensus_queue.cc:237] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:29.866367 18671 catalog_manager.cc:5495] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0de04c931e9a48fba59cd81ddd8478e3 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "0de04c931e9a48fba59cd81ddd8478e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } health_report { overall_health: HEALTHY } } }
I20260807 22:38:29.878013 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.875817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.28.82.193:45721)
0807 22:38:29.875869 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:29.875870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:29.875879 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:38:29.875946 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:29.875947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:29.875962 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:29.876024 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:38:29.876116 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.876553 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.877331 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:29.877501 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:29.877549 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:29.877734 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:38:29.877820 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:38:29.877879 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:29.877913 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 22:38:29.877929 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 22:38:29.879756 19086 raft_consensus.cc:1261] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 1 FOLLOWER]: Refusing update from remote peer 0de04c931e9a48fba59cd81ddd8478e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:29.879774 19276 raft_consensus.cc:1261] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 1 FOLLOWER]: Refusing update from remote peer 0de04c931e9a48fba59cd81ddd8478e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:29.880103 19381 consensus_queue.cc:1035] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:29.880235 19377 consensus_queue.cc:1035] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:29.882324 19384 mvcc.cc:205] Tried to move back new op lower bound from 7316038901264838656 to 7316038901210025984. Current Snapshot: MvccSnapshot[applied={T|T < 7316038901264838656}]
I20260807 22:38:29.882612 19388 mvcc.cc:205] Tried to move back new op lower bound from 7316038901264838656 to 7316038901210025984. Current Snapshot: MvccSnapshot[applied={T|T < 7316038901264838656}]
I20260807 22:38:29.884593 18671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:29.884694 18671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:29.884505 19389 mvcc.cc:205] Tried to move back new op lower bound from 7316038901264838656 to 7316038901210025984. Current Snapshot: MvccSnapshot[applied={T|T < 7316038901264838656 or (T in {7316038901264838656})}]
I20260807 22:38:29.885413 18671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Sending DeleteTablet for 3 replicas of tablet f7ae2a22d5ff485db3b94963a694ca76
I20260807 22:38:29.885739 18845 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ae2a22d5ff485db3b94963a694ca76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:29 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:52300
I20260807 22:38:29.885774 19057 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ae2a22d5ff485db3b94963a694ca76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:29 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:49858
I20260807 22:38:29.885766 19243 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ae2a22d5ff485db3b94963a694ca76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:29 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:40834
I20260807 22:38:29.885967 19399 tablet_replica.cc:330] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: stopping tablet replica
I20260807 22:38:29.886039 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 18753
I20260807 22:38:29.886060 19399 raft_consensus.cc:2229] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:38:29.886150 19399 raft_consensus.cc:2258] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:29.897600 19400 tablet_replica.cc:330] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: stopping tablet replica
I20260807 22:38:29.897795 19400 raft_consensus.cc:2229] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:29.897962 19400 raft_consensus.cc:2258] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:29.898309 19401 tablet_replica.cc:330] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: stopping tablet replica
I20260807 22:38:29.898396 19401 raft_consensus.cc:2229] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:29.898525 19401 raft_consensus.cc:2258] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:29.899116 19401 ts_tablet_manager.cc:1905] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:29.899199 19400 ts_tablet_manager.cc:1905] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:29.900671 19401 ts_tablet_manager.cc:1918] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:29.900748 19401 log.cc:1196] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/f7ae2a22d5ff485db3b94963a694ca76
I20260807 22:38:29.900990 19401 ts_tablet_manager.cc:1939] T f7ae2a22d5ff485db3b94963a694ca76 P 97cf2f6f50784e1caf5dc8e96509adb7: Deleting consensus metadata
I20260807 22:38:29.901561 19400 ts_tablet_manager.cc:1918] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:29.901625 19400 log.cc:1196] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/f7ae2a22d5ff485db3b94963a694ca76
I20260807 22:38:29.901858 19400 ts_tablet_manager.cc:1939] T f7ae2a22d5ff485db3b94963a694ca76 P 08c64815edec4d6baf964973ee1dd0fc: Deleting consensus metadata
I20260807 22:38:29.902168 18643 catalog_manager.cc:4841] TS 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317): tablet f7ae2a22d5ff485db3b94963a694ca76 (table test-table [id=81759f536c274e2a9e89d384f9ff1879]) successfully deleted
I20260807 22:38:29.902374 18643 catalog_manager.cc:4841] TS 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617): tablet f7ae2a22d5ff485db3b94963a694ca76 (table test-table [id=81759f536c274e2a9e89d384f9ff1879]) successfully deleted
W20260807 22:38:29.907996 18642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:45721 (error 108)
I20260807 22:38:29.908325 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 18965
I20260807 22:38:29.908632 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:29.908502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:45721 (local address 127.0.0.1:52312)
0807 22:38:29.908556 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:29.908592 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:45721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 22:38:29.911751 18642 catalog_manager.cc:4584] TS 0de04c931e9a48fba59cd81ddd8478e3 (127.28.82.193:45721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7ae2a22d5ff485db3b94963a694ca76: Network error: Client connection negotiation failed: client connection to 127.28.82.193:45721: connect: Connection refused (error 111)
I20260807 22:38:29.917668 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 19154
I20260807 22:38:29.927836 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 18616
I20260807 22:38:49.936748 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:35113
--webserver_interface=127.28.82.254
--webserver_port=40687
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:38:50.020221 19403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:50.020433 19403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:50.020454 19403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:50.021569 19403 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:38:50.021606 19403 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:38:50.021620 19403 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:38:50.021631 19403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:38:50.021643 19403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:50.021658 19403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:50.021670 19403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:38:50.021682 19403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:38:50.023253 19403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:35113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=40687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:50.023548 19403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:50.023800 19403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:50.033511 19416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:50.033511 19411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:50.033571 19413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:50.033769 19403 server_base.cc:1034] running on GCE node
I20260807 22:38:50.034039 19403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:50.034323 19403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:50.035494 19403 hybrid_clock.cc:648] HybridClock initialized: now 1786142330035481 us; error 32 us; skew 500 ppm
Aug 07 22:38:50 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:50.038450 19403 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:38:50.039052 19403 webserver.cc:460] Webserver started at http://127.28.82.254:40687/ using document root <none> and password file <none>
I20260807 22:38:50.039279 19403 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:50.039338 19403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:50.040870 19403 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:38:50.041555 19426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.041769 19403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:38:50.041901 19403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "e293f60604274fda9207059b5538cbe9"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "8dd4a1dec8673396b3396fc8a5eea0d5"
server_key_iv: "c24bac387d8bcecff9f7351b3d2dfb79"
server_key_version: "encryptionkey@0"
I20260807 22:38:50.042351 19403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:50.055580 19403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:50.057907 19403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:50.058096 19403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:50.062958 19403 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:35113
I20260807 22:38:50.063014 19501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:35113 every 8 connection(s)
I20260807 22:38:50.063467 19403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:38:50.065112 19502 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:38:50.065974 19502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Bootstrap starting.
I20260807 22:38:50.073263 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 19403
I20260807 22:38:50.073644 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:45721
--local_ip_for_outbound_sockets=127.28.82.193
--tserver_master_addrs=127.28.82.254:35113
--webserver_port=45439
--webserver_interface=127.28.82.193
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:50.076033 19502 log.cc:824] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:50.078109 19502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:38:50.078438 19502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Bootstrap complete.
I20260807 22:38:50.080837 19502 raft_consensus.cc:348] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:38:50.081102 19502 raft_consensus.cc:729] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e293f60604274fda9207059b5538cbe9, State: Initialized, Role: FOLLOWER
I20260807 22:38:50.081317 19502 consensus_queue.cc:260] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:38:50.081419 19502 raft_consensus.cc:388] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:38:50.081475 19502 raft_consensus.cc:482] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:38:50.081523 19502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 1 FOLLOWER]: Advancing to term 2
I20260807 22:38:50.082592 19502 raft_consensus.cc:504] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:38:50.082710 19502 leader_election.cc:302] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e293f60604274fda9207059b5538cbe9; no voters:
I20260807 22:38:50.082913 19502 leader_election.cc:288] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 22:38:50.082952 19506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 22:38:50.083071 19506 raft_consensus.cc:686] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 2 LEADER]: Becoming Leader. State: Replica: e293f60604274fda9207059b5538cbe9, State: Running, Role: LEADER
I20260807 22:38:50.083185 19502 sys_catalog.cc:564] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:38:50.083159 19506 consensus_queue.cc:237] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:38:50.083465 19506 sys_catalog.cc:455] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e293f60604274fda9207059b5538cbe9. Latest consensus state: current_term: 2 leader_uuid: "e293f60604274fda9207059b5538cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } } }
I20260807 22:38:50.083539 19506 sys_catalog.cc:458] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:50.083493 19507 sys_catalog.cc:455] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e293f60604274fda9207059b5538cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } } }
I20260807 22:38:50.083705 19507 sys_catalog.cc:458] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:38:50.084314 19510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:38:50.085278 19510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:38:50.085412 19510 catalog_manager.cc:1259] Loaded cluster ID: 62e893366b4649309dcbb5b9fea9ca71
I20260807 22:38:50.085455 19510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:38:50.086793 19510 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:38:50.086977 19510 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Loaded TSK: 0
I20260807 22:38:50.097002 19510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Generated new TSK 1
I20260807 22:38:50.097100 19510 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:38:50.163991 19504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:50.164204 19504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:50.164228 19504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:50.165423 19504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:50.165469 19504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:50.165495 19504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:38:50.167012 19504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:45721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=45439
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:50.167295 19504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:50.167562 19504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:50.172786 19542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:50.172760 19540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:50.172742 19539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:50.173035 19504 server_base.cc:1034] running on GCE node
I20260807 22:38:50.173312 19504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:50.173583 19504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:50.174757 19504 hybrid_clock.cc:648] HybridClock initialized: now 1786142330174739 us; error 35 us; skew 500 ppm
Aug 07 22:38:50 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:50.177858 19504 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:38:50.178354 19504 webserver.cc:460] Webserver started at http://127.28.82.193:45439/ using document root <none> and password file <none>
I20260807 22:38:50.178647 19504 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:50.178815 19504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:50.180307 19504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:50.181097 19551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.181304 19504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:50.181412 19504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "0de04c931e9a48fba59cd81ddd8478e3"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "abc4696c3bad19436d3cd61b2218a081"
server_key_iv: "62a46e460de126d0abd22a5a06cb8e6e"
server_key_version: "encryptionkey@0"
I20260807 22:38:50.181783 19504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:50.200985 19504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:50.203634 19504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:50.203838 19504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:50.204125 19504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:50.204757 19562 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 22:38:50.206399 19504 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:38:50.206514 19504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 22:38:50.206573 19504 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:38:50.207350 19504 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:38:50.207404 19504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 22:38:50.207454 19562 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Bootstrap starting.
I20260807 22:38:50.216204 19504 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:45721
I20260807 22:38:50.216331 19728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:45721 every 8 connection(s)
I20260807 22:38:50.216691 19504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:38:50.219594 19562 log.cc:824] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:50.220507 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 19504
I20260807 22:38:50.220835 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:34317
--local_ip_for_outbound_sockets=127.28.82.194
--tserver_master_addrs=127.28.82.254:35113
--webserver_port=43313
--webserver_interface=127.28.82.194
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:38:50.222301 19562 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:38:50.222661 19562 tablet_bootstrap.cc:492] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Bootstrap complete.
I20260807 22:38:50.223974 19562 ts_tablet_manager.cc:1397] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20260807 22:38:50.225006 19562 raft_consensus.cc:348] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.225356 19562 raft_consensus.cc:729] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de04c931e9a48fba59cd81ddd8478e3, State: Initialized, Role: FOLLOWER
I20260807 22:38:50.225499 19562 consensus_queue.cc:260] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
Aug 07 22:38:50 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:50.225776 19562 ts_tablet_manager.cc:1428] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 22:38:50.230216 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.217717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:35519 (local address 127.28.82.254:35113)
0807 22:38:50.217962 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.217972 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.218027 (+ 55us) server_negotiation.cc:407] Connection header received
0807 22:38:50.218643 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.218653 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.218723 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.218885 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:50.219561 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.220975 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.222536 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.223151 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.223271 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.226362 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:50.226366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:50.226366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:50.226377 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:50.228144 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.228689 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.228691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.228692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.228783 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.229126 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.229127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.229127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.229475 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:50.229548 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.229945 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.230054 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":110,"threads_started":1}
I20260807 22:38:50.230342 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.217532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:35519)
0807 22:38:50.217773 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:50.217830 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:50.218370 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:50.218921 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:50.218925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:50.219226 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:50.219430 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.219435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.221121 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.221122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.222397 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.222400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.223276 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.223278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.223444 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.224157 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:50.224169 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:50.226244 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:50.228233 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:50.228241 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:50.228243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:50.228592 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:50.228868 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:50.228869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:50.228869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:50.229025 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:50.229574 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:50.229582 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:50.229767 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 22:38:50.229896 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 22:38:50.230091 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":124,"threads_started":1}
I20260807 22:38:50.231156 19729 heartbeater.cc:347] Connected to a master server at 127.28.82.254:35113
I20260807 22:38:50.231283 19729 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:50.231462 19729 heartbeater.cc:511] Master 127.28.82.254:35113 requested a full tablet report, sending...
I20260807 22:38:50.232172 19454 ts_manager.cc:194] Registered new tserver with Master: 0de04c931e9a48fba59cd81ddd8478e3 (127.28.82.193:45721)
I20260807 22:38:50.233301 19454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:35519
I20260807 22:38:50.234179 19729 heartbeater.cc:503] Master 127.28.82.254:35113 was elected leader, sending a full tablet report...
Aug 07 22:38:50 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.193@KRBTEST.COM
I20260807 22:38:50.241531 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.232918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.28.82.193:45721)
0807 22:38:50.233070 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.233074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.233084 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:38:50.233114 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.233116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.233128 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.233200 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:50.233880 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.234889 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.236068 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.236377 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.236439 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.238627 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:50.238635 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:50.238636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:50.238645 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:50.240237 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.240638 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.240641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.240641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.240703 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.240921 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.240922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.240922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.241060 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:50.241164 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.241353 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.241444 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260807 22:38:50.241531 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.232718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:45721 (local address 127.0.0.1:60936)
0807 22:38:50.232905 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:50.232948 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:50.233019 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:50.233222 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:50.233230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:50.233720 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:50.233788 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.233790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.234971 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.234972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.235976 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.235979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.236461 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.236462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.236543 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.236994 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:50.236999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:50.238541 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:50.240336 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:50.240343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:50.240344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:50.240550 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:50.240782 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:50.240782 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:50.240783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:50.240850 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:50.241136 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:50.241143 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:50.241197 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 22:38:50.241300 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 22:38:50.241430 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260807 22:38:50.241986 19626 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ae2a22d5ff485db3b94963a694ca76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:29 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:60936
I20260807 22:38:50.242630 19741 tablet_replica.cc:330] stopping tablet replica
I20260807 22:38:50.242760 19741 raft_consensus.cc:2229] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:50.242858 19741 raft_consensus.cc:2258] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:50.243177 19741 ts_tablet_manager.cc:1905] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:50.245333 19741 ts_tablet_manager.cc:1918] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:50.245429 19741 log.cc:1196] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal/wals/f7ae2a22d5ff485db3b94963a694ca76
I20260807 22:38:50.245671 19741 ts_tablet_manager.cc:1939] T f7ae2a22d5ff485db3b94963a694ca76 P 0de04c931e9a48fba59cd81ddd8478e3: Deleting consensus metadata
I20260807 22:38:50.246233 19427 catalog_manager.cc:4841] TS 0de04c931e9a48fba59cd81ddd8478e3 (127.28.82.193:45721): tablet f7ae2a22d5ff485db3b94963a694ca76 (table test-table [id=81759f536c274e2a9e89d384f9ff1879]) successfully deleted
W20260807 22:38:50.310882 19737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:50.311085 19737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:50.311110 19737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:50.312217 19737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:50.312253 19737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:50.312282 19737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:38:50.313674 19737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:34317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=43313
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:50.313938 19737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:50.314188 19737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:50.319444 19752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:50.319460 19754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:50.319623 19737 server_base.cc:1034] running on GCE node
W20260807 22:38:50.319444 19759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:50.319994 19737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:50.320248 19737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:50.321424 19737 hybrid_clock.cc:648] HybridClock initialized: now 1786142330321404 us; error 35 us; skew 500 ppm
Aug 07 22:38:50 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:50.324131 19737 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:38:50.324558 19737 webserver.cc:460] Webserver started at http://127.28.82.194:43313/ using document root <none> and password file <none>
I20260807 22:38:50.324841 19737 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:50.324937 19737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:50.326251 19737 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:50.326910 19774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.327083 19737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.327185 19737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "97cf2f6f50784e1caf5dc8e96509adb7"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "c3faf3c608022bb6a1b3408c1420edcf"
server_key_iv: "9b6ea954af49521b2aa572ef0f7db370"
server_key_version: "encryptionkey@0"
I20260807 22:38:50.327538 19737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:50.362782 19737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:50.365118 19737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:50.365288 19737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:50.365543 19737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:50.365878 19737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:50.365934 19737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.365981 19737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:50.366014 19737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.372081 19737 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:34317
I20260807 22:38:50.372128 19941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:34317 every 8 connection(s)
I20260807 22:38:50.372458 19737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:38:50.377964 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 19737
I20260807 22:38:50.378293 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:43617
--local_ip_for_outbound_sockets=127.28.82.195
--tserver_master_addrs=127.28.82.254:35113
--webserver_port=45173
--webserver_interface=127.28.82.195
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 22:38:50 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:50.382535 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.373101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:48073 (local address 127.28.82.254:35113)
0807 22:38:50.373152 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.373158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.373355 (+ 197us) server_negotiation.cc:407] Connection header received
0807 22:38:50.373934 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.373935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.373950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.374004 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:50.375092 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.375599 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.376612 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.376833 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.376924 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.379381 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:50.379382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:50.379382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:50.379390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:50.381083 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.381500 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.381501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.381502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.381570 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.381811 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.381812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.381813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.381961 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:50.382036 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.382271 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.382373 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:38:50.382568 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.373063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:48073)
0807 22:38:50.373231 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:50.373274 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:50.373815 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:50.374113 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:50.374118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:50.374794 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:50.374987 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.374991 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.375706 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.375707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.376501 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.376503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.376951 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.376953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.377078 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.377505 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:50.377513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:50.379257 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:50.381172 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:50.381179 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:50.381181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:50.381422 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:50.381643 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:50.381644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:50.381644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:50.381744 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:50.382030 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:50.382034 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:50.382142 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 22:38:50.382222 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 22:38:50.382354 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260807 22:38:50.383131 19942 heartbeater.cc:347] Connected to a master server at 127.28.82.254:35113
I20260807 22:38:50.383239 19942 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:50.383432 19942 heartbeater.cc:511] Master 127.28.82.254:35113 requested a full tablet report, sending...
I20260807 22:38:50.383906 19454 ts_manager.cc:194] Registered new tserver with Master: 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
I20260807 22:38:50.384310 19454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:48073
W20260807 22:38:50.463258 19946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:38:50.463496 19946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:38:50.463536 19946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:38:50.464802 19946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:38:50.464866 19946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:38:50.464911 19946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:38:50.466383 19946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:43617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=45173
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:38:50.466677 19946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:38:50.466912 19946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:38:50.472273 19959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:38:50.472250 19957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:50.472293 19946 server_base.cc:1034] running on GCE node
W20260807 22:38:50.472259 19956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:38:50.472725 19946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:38:50.472927 19946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:38:50.474080 19946 hybrid_clock.cc:648] HybridClock initialized: now 1786142330474047 us; error 44 us; skew 500 ppm
Aug 07 22:38:50 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:38:50.476864 19946 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:38:50.477337 19946 webserver.cc:460] Webserver started at http://127.28.82.195:45173/ using document root <none> and password file <none>
I20260807 22:38:50.477594 19946 fs_manager.cc:359] Metadata directory not provided
I20260807 22:38:50.477682 19946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:38:50.479218 19946 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:50.479880 19968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.480073 19946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:50.480178 19946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "08c64815edec4d6baf964973ee1dd0fc"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "23cffd11a33667a01e7826de616acd61"
server_key_iv: "6fd871eedb31629eb2acd1155568da1b"
server_key_version: "encryptionkey@0"
I20260807 22:38:50.480531 19946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:38:50.496955 19946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:38:50.499403 19946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:38:50.499572 19946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:38:50.499832 19946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:38:50.500161 19946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:38:50.500217 19946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.500252 19946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:38:50.500278 19946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:38:50.506245 19946 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:43617
I20260807 22:38:50.506300 20143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:43617 every 8 connection(s)
I20260807 22:38:50.506695 19946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:38:50 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:38:50.515552 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 19946
I20260807 22:38:50.515808 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.507405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:60837 (local address 127.28.82.254:35113)
0807 22:38:50.507476 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.507478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.507718 (+ 240us) server_negotiation.cc:407] Connection header received
0807 22:38:50.508184 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.508185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.508197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.508245 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:50.509133 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.509625 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.510643 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.510833 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.510889 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.513018 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:50.513019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:50.513020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:50.513027 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:50.514323 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.514792 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.514793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.514794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.514857 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.515087 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.515088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.515088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.515202 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:50.515255 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.515581 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.515692 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:38:50.516045 20147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.507418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:60837)
0807 22:38:50.507591 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:50.507628 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:50.508100 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:50.508303 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:50.508306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:50.508875 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:38:50.509058 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.509062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.509708 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.509709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.510560 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.510562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.510908 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.510909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.511007 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.511461 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:38:50.511470 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:38:50.512880 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:38:50.514400 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:50.514407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:50.514409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:50.514717 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:50.514925 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:38:50.514926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:38:50.514926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:38:50.515030 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:38:50.515260 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:38:50.515265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:38:50.515437 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 22:38:50.515559 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 22:38:50.515781 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260807 22:38:50.516631 20144 heartbeater.cc:347] Connected to a master server at 127.28.82.254:35113
I20260807 22:38:50.516721 20144 heartbeater.cc:464] Registering TS with master...
I20260807 22:38:50.516945 20144 heartbeater.cc:511] Master 127.28.82.254:35113 requested a full tablet report, sending...
I20260807 22:38:50.517374 19454 ts_manager.cc:194] Registered new tserver with Master: 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617)
I20260807 22:38:50.517750 19454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:60837
I20260807 22:38:50.522712 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.516135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.28.82.254:35113)
0807 22:38:50.516295 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.516297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.516305 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:38:50.516322 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.516323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.516333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.516397 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:50.517235 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.517783 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.518924 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.519123 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.519197 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.520357 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:50.520358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:50.520359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:50.520367 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:50.521509 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.521896 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.521898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.521898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.521955 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.522175 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.522176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.522176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.522294 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:50.522499 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.522539 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.522616 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 22:38:50.523561 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:38:50.525906 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.523901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.28.82.254:35113)
0807 22:38:50.523938 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.523940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.524043 (+ 103us) server_negotiation.cc:407] Connection header received
0807 22:38:50.524087 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.524088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.524120 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.524178 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:38:50.524246 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.524698 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.525361 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.525535 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.525585 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.525724 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:38:50.525763 (+ 39us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 22:38:50.525866 (+ 103us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46798: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 22:38:50.526014 19735 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46798: token expired
W20260807 22:38:50.526175 19351 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.28.82.254:35113)
I20260807 22:38:50.532047 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.526461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46802 (local address 127.28.82.254:35113)
0807 22:38:50.526510 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.526512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.526520 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:38:50.526585 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.526585 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.526592 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.526668 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:38:50.527092 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.527518 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.528184 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.528349 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.528400 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.529503 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:38:50.529504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:38:50.529504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:38:50.529511 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:38:50.530940 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.531271 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.531271 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.531272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.531325 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:38:50.531591 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:38:50.531592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:38:50.531592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:38:50.531739 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:38:50.531828 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.531937 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.531995 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 22:38:50.533205 19454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:38:50.534857 19454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:38:50.536998 19626 tablet_service.cc:1467] Processing CreateTablet for tablet 33e07b888a90418e8b681b39d3f6f7a1 (DEFAULT_TABLE table=test-table [id=30a265bef9d14854bf3cb939537c3763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:50.537168 19626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33e07b888a90418e8b681b39d3f6f7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:50.538861 19562 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Bootstrap starting.
I20260807 22:38:50.539213 19562 tablet_bootstrap.cc:654] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:50.539824 19562 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: No bootstrap required, opened a new log
I20260807 22:38:50.539873 19562 ts_tablet_manager.cc:1397] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:50.540253 19562 raft_consensus.cc:348] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.540343 19562 raft_consensus.cc:374] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:50.540386 19562 raft_consensus.cc:729] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de04c931e9a48fba59cd81ddd8478e3, State: Initialized, Role: FOLLOWER
I20260807 22:38:50.540437 19562 consensus_queue.cc:260] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.540524 19562 ts_tablet_manager.cc:1428] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 22:38:50.541335 20154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.537336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43894 (local address 127.28.82.194:34317)
0807 22:38:50.537560 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.537565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.537575 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:38:50.537625 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.537627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.537643 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.537729 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:50.537908 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.538994 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.540079 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.540846 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.540917 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.540975 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.541208 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.541250 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":110,"threads_started":1}
I20260807 22:38:50.541322 20152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.536872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:34317 (local address 127.0.0.1:43894)
0807 22:38:50.536967 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:50.536976 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:50.537017 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:50.537748 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:50.537749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:50.537752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:50.537794 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.537795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.539057 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.539058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.540010 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.540013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.541066 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.541066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.541132 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.541136 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:38:50.541190 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 22:38:50.541230 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":65,"threads_started":1}
I20260807 22:38:50.541390 20153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.537014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33656 (local address 127.28.82.195:43617)
0807 22:38:50.537174 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.537179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.537191 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:38:50.537234 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.537236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.537252 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.537517 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:50.537605 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.538556 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.540170 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.540927 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.541029 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.541084 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.541262 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.541305 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260807 22:38:50.541386 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.536756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:43617 (local address 127.0.0.1:33656)
0807 22:38:50.536811 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:50.536823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:50.536861 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:50.537365 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:50.537367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:50.537369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:50.537420 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.537421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.538645 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.538646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.539778 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.539782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.541031 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.541032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.541106 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.541129 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 22:38:50.541212 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:38:50.541268 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 22:38:50.542363 20035 tablet_service.cc:1467] Processing CreateTablet for tablet 33e07b888a90418e8b681b39d3f6f7a1 (DEFAULT_TABLE table=test-table [id=30a265bef9d14854bf3cb939537c3763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:50.542431 19845 tablet_service.cc:1467] Processing CreateTablet for tablet 33e07b888a90418e8b681b39d3f6f7a1 (DEFAULT_TABLE table=test-table [id=30a265bef9d14854bf3cb939537c3763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:38:50.542744 20035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33e07b888a90418e8b681b39d3f6f7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:50.542861 19845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33e07b888a90418e8b681b39d3f6f7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:38:50.545585 20157 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Bootstrap starting.
I20260807 22:38:50.545720 20159 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Bootstrap starting.
I20260807 22:38:50.546330 20157 tablet_bootstrap.cc:654] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:50.546368 20159 tablet_bootstrap.cc:654] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Neither blocks nor log segments found. Creating new log.
I20260807 22:38:50.546905 20159 log.cc:824] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:50.546909 20157 log.cc:824] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:38:50.547631 20159 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: No bootstrap required, opened a new log
I20260807 22:38:50.547633 20157 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: No bootstrap required, opened a new log
I20260807 22:38:50.547688 20159 ts_tablet_manager.cc:1397] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:50.547712 20157 ts_tablet_manager.cc:1397] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:50.549280 20159 raft_consensus.cc:348] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.549417 20159 raft_consensus.cc:374] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:50.549439 20159 raft_consensus.cc:729] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c64815edec4d6baf964973ee1dd0fc, State: Initialized, Role: FOLLOWER
I20260807 22:38:50.549329 20157 raft_consensus.cc:348] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.549501 20157 raft_consensus.cc:374] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:38:50.549551 20157 raft_consensus.cc:729] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97cf2f6f50784e1caf5dc8e96509adb7, State: Initialized, Role: FOLLOWER
I20260807 22:38:50.549559 20159 consensus_queue.cc:260] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.549674 20157 consensus_queue.cc:260] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.549926 20159 ts_tablet_manager.cc:1428] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:50.549997 20144 heartbeater.cc:503] Master 127.28.82.254:35113 was elected leader, sending a full tablet report...
I20260807 22:38:50.549993 20157 ts_tablet_manager.cc:1428] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:38:50.550028 19942 heartbeater.cc:503] Master 127.28.82.254:35113 was elected leader, sending a full tablet report...
W20260807 22:38:50.623139 19943 tablet.cc:2369] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:38:50.717622 19730 tablet.cc:2369] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:38:50.757335 20146 tablet.cc:2369] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:38:50.887667 19738 raft_consensus.cc:482] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:38:50.887821 19738 raft_consensus.cc:504] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.888175 19738 leader_election.cc:288] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617), 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
I20260807 22:38:50.891391 20153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.888497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:36739 (local address 127.28.82.195:43617)
0807 22:38:50.888578 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.888581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.888616 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:38:50.888787 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.888788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.888800 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.888857 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:50.889103 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.889607 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.890601 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.890983 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.891084 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.891094 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.891278 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.891322 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 22:38:50.891388 20154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.888662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:51951 (local address 127.28.82.194:34317)
0807 22:38:50.888706 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.888708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.888718 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:38:50.888858 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.888860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.888870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.888920 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:38:50.889133 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.889613 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.890629 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.890991 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.891081 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.891100 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.891271 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.891311 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:38:50.891420 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.888381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:43617 (local address 127.28.82.193:36739)
0807 22:38:50.888538 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:50.888552 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:50.888649 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:50.888922 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:50.888923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:50.888926 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:50.888995 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.888997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.889701 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.889702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.890500 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.890502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.891087 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.891089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.891163 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.891190 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 22:38:50.891262 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 22:38:50.891313 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":5,"thread_start_us":77,"threads_started":1}
I20260807 22:38:50.891420 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.888578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:34317 (local address 127.28.82.193:51951)
0807 22:38:50.888662 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:50.888674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:38:50.888731 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:38:50.888994 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:38:50.888996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:38:50.888998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:38:50.889069 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.889071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.889720 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.889721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.890529 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:38:50.890532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.891092 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.891093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:38:50.891159 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.891193 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 22:38:50.891263 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:38:50.891311 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":1,"thread_start_us":39,"threads_started":1}
I20260807 22:38:50.891816 20069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33e07b888a90418e8b681b39d3f6f7a1" candidate_uuid: "0de04c931e9a48fba59cd81ddd8478e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c64815edec4d6baf964973ee1dd0fc" is_pre_election: true
I20260807 22:38:50.891999 20069 raft_consensus.cc:2454] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0de04c931e9a48fba59cd81ddd8478e3 in term 0.
I20260807 22:38:50.891966 19875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33e07b888a90418e8b681b39d3f6f7a1" candidate_uuid: "0de04c931e9a48fba59cd81ddd8478e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" is_pre_election: true
I20260807 22:38:50.892128 19875 raft_consensus.cc:2454] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0de04c931e9a48fba59cd81ddd8478e3 in term 0.
I20260807 22:38:50.892243 19553 leader_election.cc:302] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c64815edec4d6baf964973ee1dd0fc, 0de04c931e9a48fba59cd81ddd8478e3; no voters:
I20260807 22:38:50.892376 19738 raft_consensus.cc:2781] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:38:50.892434 19738 raft_consensus.cc:482] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:38:50.892458 19738 raft_consensus.cc:3037] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:50.893126 19738 raft_consensus.cc:504] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.893270 19738 leader_election.cc:288] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 1 election: Requested vote from peers 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617), 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
I20260807 22:38:50.893410 20069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33e07b888a90418e8b681b39d3f6f7a1" candidate_uuid: "0de04c931e9a48fba59cd81ddd8478e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c64815edec4d6baf964973ee1dd0fc"
I20260807 22:38:50.893455 19875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33e07b888a90418e8b681b39d3f6f7a1" candidate_uuid: "0de04c931e9a48fba59cd81ddd8478e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97cf2f6f50784e1caf5dc8e96509adb7"
I20260807 22:38:50.893493 20069 raft_consensus.cc:3037] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:50.893545 19875 raft_consensus.cc:3037] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:38:50.894227 20069 raft_consensus.cc:2454] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0de04c931e9a48fba59cd81ddd8478e3 in term 1.
I20260807 22:38:50.894322 19875 raft_consensus.cc:2454] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0de04c931e9a48fba59cd81ddd8478e3 in term 1.
I20260807 22:38:50.894418 19553 leader_election.cc:302] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c64815edec4d6baf964973ee1dd0fc, 0de04c931e9a48fba59cd81ddd8478e3; no voters:
I20260807 22:38:50.894547 19738 raft_consensus.cc:2781] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:38:50.894734 19738 raft_consensus.cc:686] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 LEADER]: Becoming Leader. State: Replica: 0de04c931e9a48fba59cd81ddd8478e3, State: Running, Role: LEADER
I20260807 22:38:50.894824 19738 consensus_queue.cc:237] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:38:50.895620 19453 catalog_manager.cc:5495] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0de04c931e9a48fba59cd81ddd8478e3 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "0de04c931e9a48fba59cd81ddd8478e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } health_report { overall_health: HEALTHY } } }
I20260807 22:38:50.981133 20173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:50.978557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.28.82.193:45721)
0807 22:38:50.978814 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 22:38:50.978816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:38:50.978824 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:38:50.978867 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:38:50.978867 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:38:50.978888 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:38:50.978962 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:38:50.979111 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.979599 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.980342 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:38:50.980531 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:38:50.980586 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:38:50.980792 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:38:50.980894 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:38:50.980948 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:38:50.981003 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 22:38:50.981020 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":148,"threads_started":1}
I20260807 22:38:50.982782 20069 raft_consensus.cc:1261] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 1 FOLLOWER]: Refusing update from remote peer 0de04c931e9a48fba59cd81ddd8478e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:50.982818 19875 raft_consensus.cc:1261] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 1 FOLLOWER]: Refusing update from remote peer 0de04c931e9a48fba59cd81ddd8478e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:38:50.983191 20170 consensus_queue.cc:1035] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:50.983394 19738 consensus_queue.cc:1035] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:38:50.985945 20174 mvcc.cc:205] Tried to move back new op lower bound from 7316038987702681600 to 7316038987345936384. Current Snapshot: MvccSnapshot[applied={T|T < 7316038987702681600}]
I20260807 22:38:50.986508 20175 mvcc.cc:205] Tried to move back new op lower bound from 7316038987702681600 to 7316038987345936384. Current Snapshot: MvccSnapshot[applied={T|T < 7316038987702681600}]
I20260807 22:38:50.987250 20178 mvcc.cc:205] Tried to move back new op lower bound from 7316038987702681600 to 7316038987345936384. Current Snapshot: MvccSnapshot[applied={T|T < 7316038987702681600 or (T in {7316038987702681600})}]
I20260807 22:38:50.988711 19453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:50.988801 19453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:38:50.989544 19453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Sending DeleteTablet for 3 replicas of tablet 33e07b888a90418e8b681b39d3f6f7a1
I20260807 22:38:50.989910 20035 tablet_service.cc:1514] Processing DeleteTablet for tablet 33e07b888a90418e8b681b39d3f6f7a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:50 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:33656
I20260807 22:38:50.989889 19845 tablet_service.cc:1514] Processing DeleteTablet for tablet 33e07b888a90418e8b681b39d3f6f7a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:38:50 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:43894
I20260807 22:38:50.990154 20192 tablet_replica.cc:330] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: stopping tablet replica
I20260807 22:38:50.990154 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 19504
I20260807 22:38:50.990269 20192 raft_consensus.cc:2229] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:51.006124 20192 raft_consensus.cc:2258] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:51.006848 20192 ts_tablet_manager.cc:1905] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:51.009159 20192 ts_tablet_manager.cc:1918] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:51.009254 20192 log.cc:1196] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/33e07b888a90418e8b681b39d3f6f7a1
I20260807 22:38:51.009452 20191 tablet_replica.cc:330] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: stopping tablet replica
I20260807 22:38:51.009564 20192 ts_tablet_manager.cc:1939] T 33e07b888a90418e8b681b39d3f6f7a1 P 97cf2f6f50784e1caf5dc8e96509adb7: Deleting consensus metadata
I20260807 22:38:51.009601 20191 raft_consensus.cc:2229] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:38:51.009732 20191 raft_consensus.cc:2258] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:38:51.010077 19428 catalog_manager.cc:4841] TS 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317): tablet 33e07b888a90418e8b681b39d3f6f7a1 (table test-table [id=30a265bef9d14854bf3cb939537c3763]) successfully deleted
I20260807 22:38:51.010212 20191 ts_tablet_manager.cc:1905] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:38:51.011657 20191 ts_tablet_manager.cc:1918] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:38:51.011746 20191 log.cc:1196] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/33e07b888a90418e8b681b39d3f6f7a1
I20260807 22:38:51.012070 20191 ts_tablet_manager.cc:1939] T 33e07b888a90418e8b681b39d3f6f7a1 P 08c64815edec4d6baf964973ee1dd0fc: Deleting consensus metadata
I20260807 22:38:51.012526 19428 catalog_manager.cc:4841] TS 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617): tablet 33e07b888a90418e8b681b39d3f6f7a1 (table test-table [id=30a265bef9d14854bf3cb939537c3763]) successfully deleted
W20260807 22:38:51.012748 19427 connection.cc:535] client connection to 127.28.82.193:45721 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 22:38:51.012884 19427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:38:51.013423 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:38:51.013299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:45721 (local address 127.0.0.1:60966)
0807 22:38:51.013358 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 22:38:51.013396 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:45721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 22:38:51.013545 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 19737
W20260807 22:38:51.013610 19427 catalog_manager.cc:4584] TS 0de04c931e9a48fba59cd81ddd8478e3 (127.28.82.193:45721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33e07b888a90418e8b681b39d3f6f7a1: Network error: Client connection negotiation failed: client connection to 127.28.82.193:45721: connect: Connection refused (error 111)
I20260807 22:38:51.036660 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 19946
I20260807 22:38:51.046301 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 19403
I20260807 22:39:11.055739 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:35113
--webserver_interface=127.28.82.254
--webserver_port=40687
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:11.139394 20199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:11.139648 20199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:11.139685 20199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:11.140836 20199 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 22:39:11.140914 20199 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 22:39:11.140935 20199 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 22:39:11.140955 20199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:11.140987 20199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:11.141021 20199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:11.141044 20199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:11.141072 20199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:11.142587 20199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:35113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=40687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:11.142872 20199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:11.143115 20199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:11.148371 20210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:11.148391 20212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:11.148473 20199 server_base.cc:1034] running on GCE node
W20260807 22:39:11.148425 20207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:11.148854 20199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:11.149125 20199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:11.150307 20199 hybrid_clock.cc:648] HybridClock initialized: now 1786142351150297 us; error 52 us; skew 500 ppm
Aug 07 22:39:11 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142351, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:11.152925 20199 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:39:11.153371 20199 webserver.cc:460] Webserver started at http://127.28.82.254:40687/ using document root <none> and password file <none>
I20260807 22:39:11.153623 20199 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:11.153697 20199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:11.155154 20199 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 22:39:11.155841 20228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:11.156018 20199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:39:11.156157 20199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "e293f60604274fda9207059b5538cbe9"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "8dd4a1dec8673396b3396fc8a5eea0d5"
server_key_iv: "c24bac387d8bcecff9f7351b3d2dfb79"
server_key_version: "encryptionkey@0"
I20260807 22:39:11.156535 20199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:11.171444 20199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:11.173332 20199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:11.173508 20199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:11.177624 20199 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:35113
I20260807 22:39:11.177662 20300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:35113 every 8 connection(s)
I20260807 22:39:11.178035 20199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:11.179572 20302 sys_catalog.cc:263] Verifying existing consensus state
I20260807 22:39:11.180240 20302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Bootstrap starting.
I20260807 22:39:11.182199 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 20199
I20260807 22:39:11.182555 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:45721
--local_ip_for_outbound_sockets=127.28.82.193
--tserver_master_addrs=127.28.82.254:35113
--webserver_port=45439
--webserver_interface=127.28.82.193
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:39:11.190201 20302 log.cc:824] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:11.192415 20302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:39:11.192746 20302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Bootstrap complete.
I20260807 22:39:11.195174 20302 raft_consensus.cc:348] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:39:11.195387 20302 raft_consensus.cc:729] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e293f60604274fda9207059b5538cbe9, State: Initialized, Role: FOLLOWER
I20260807 22:39:11.195555 20302 consensus_queue.cc:260] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:39:11.195616 20302 raft_consensus.cc:388] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:11.195680 20302 raft_consensus.cc:482] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:11.195742 20302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 2 FOLLOWER]: Advancing to term 3
I20260807 22:39:11.196740 20302 raft_consensus.cc:504] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:39:11.196853 20302 leader_election.cc:302] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e293f60604274fda9207059b5538cbe9; no voters:
I20260807 22:39:11.197072 20302 leader_election.cc:288] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 22:39:11.197124 20306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 22:39:11.197284 20306 raft_consensus.cc:686] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [term 3 LEADER]: Becoming Leader. State: Replica: e293f60604274fda9207059b5538cbe9, State: Running, Role: LEADER
I20260807 22:39:11.197415 20302 sys_catalog.cc:564] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:11.197427 20306 consensus_queue.cc:237] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } }
I20260807 22:39:11.197760 20308 sys_catalog.cc:455] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e293f60604274fda9207059b5538cbe9. Latest consensus state: current_term: 3 leader_uuid: "e293f60604274fda9207059b5538cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } } }
I20260807 22:39:11.197901 20308 sys_catalog.cc:458] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:11.197763 20307 sys_catalog.cc:455] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e293f60604274fda9207059b5538cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e293f60604274fda9207059b5538cbe9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 35113 } } }
I20260807 22:39:11.198086 20307 sys_catalog.cc:458] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:11.198220 20316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:11.199082 20316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:11.199198 20316 catalog_manager.cc:1259] Loaded cluster ID: 62e893366b4649309dcbb5b9fea9ca71
I20260807 22:39:11.199247 20316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:11.200412 20316 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:11.200601 20316 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Loaded TSK: 0
I20260807 22:39:11.200654 20316 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e293f60604274fda9207059b5538cbe9: Loaded TSK: 1
F20260807 22:39:11.200870 20316 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fab6ac949ad google::LogMessage::Fail() at ??:0
@ 0x7fab6ac9788a google::LogMessage::SendToLog() at ??:0
@ 0x7fab6ac944de google::LogMessage::Flush() at ??:0
@ 0x7fab6ac95a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fab6d401104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fab6d3fed41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fab72bee1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fab72bee48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fab72bee594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fab72bb4c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fab72bbead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fab6cb620d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fab6cb57954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fab6d9bb6db start_thread at ??:0
@ 0x7fab6979261f clone at ??:0
W20260807 22:39:11.297259 20304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:11.297537 20304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:11.297580 20304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:11.299330 20304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:11.299393 20304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:11.299435 20304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:39:11.301651 20304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:45721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=45439
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:11.301898 20304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:11.302161 20304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:11.307618 20365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:11.307623 20362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:11.307623 20363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:11.308081 20304 server_base.cc:1034] running on GCE node
I20260807 22:39:11.308321 20304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:11.308584 20304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:11.309757 20304 hybrid_clock.cc:648] HybridClock initialized: now 1786142351309745 us; error 33 us; skew 500 ppm
Aug 07 22:39:11 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142351, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:11.312480 20304 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:39:11.312901 20304 webserver.cc:460] Webserver started at http://127.28.82.193:45439/ using document root <none> and password file <none>
I20260807 22:39:11.313146 20304 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:11.313227 20304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:11.314636 20304 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:11.315379 20373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:11.315560 20304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:11.315666 20304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "0de04c931e9a48fba59cd81ddd8478e3"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "abc4696c3bad19436d3cd61b2218a081"
server_key_iv: "62a46e460de126d0abd22a5a06cb8e6e"
server_key_version: "encryptionkey@0"
I20260807 22:39:11.316008 20304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:11.329995 20304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:11.332351 20304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:11.332515 20304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:11.332741 20304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:11.333180 20385 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 22:39:11.335114 20304 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 22:39:11.335180 20304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 22:39:11.335237 20304 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 22:39:11.335842 20304 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 22:39:11.335901 20304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 22:39:11.335974 20385 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Bootstrap starting.
I20260807 22:39:11.343382 20304 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:45721
I20260807 22:39:11.343520 20533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:45721 every 8 connection(s)
I20260807 22:39:11.343802 20304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:39:11.348026 20385 log.cc:824] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:11.349046 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 20304
I20260807 22:39:11.349383 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:34317
--local_ip_for_outbound_sockets=127.28.82.194
--tserver_master_addrs=127.28.82.254:35113
--webserver_port=43313
--webserver_interface=127.28.82.194
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:39:11.350934 20385 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 22:39:11.351258 20385 tablet_bootstrap.cc:492] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Bootstrap complete.
I20260807 22:39:11.352748 20385 ts_tablet_manager.cc:1397] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.004s
I20260807 22:39:11.354411 20385 raft_consensus.cc:348] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:39:11.354816 20385 raft_consensus.cc:729] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de04c931e9a48fba59cd81ddd8478e3, State: Initialized, Role: FOLLOWER
I20260807 22:39:11.355008 20385 consensus_queue.cc:260] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:39:11.355315 20385 ts_tablet_manager.cc:1428] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260807 22:39:11.424214 20537 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:39:11.344499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:37591)
0807 22:39:11.344679 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.344728 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:11.345154 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:11.424002 (+ 78848us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
W20260807 22:39:11.424455 20534 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:35113 (0 consecutive failures): Network error: Failed to ping master at 127.28.82.254:35113: Client connection negotiation failed: client connection to 127.28.82.254:35113: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 22:39:11.424815 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:11.424683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:35947)
0807 22:39:11.424744 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.424794 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 22:39:11.425222 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:11.425127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:51215)
0807 22:39:11.425178 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.425204 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 22:39:11.425366 20534 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 22:39:11.438027 20540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:11.438254 20540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:11.438301 20540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:11.439512 20540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:11.439580 20540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:11.439625 20540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:39:11.441104 20540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:34317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=43313
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:11.441390 20540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:11.441625 20540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:11.446816 20547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:11.446808 20551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:11.446807 20549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:11.447021 20540 server_base.cc:1034] running on GCE node
I20260807 22:39:11.447340 20540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:11.447587 20540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:11.448801 20540 hybrid_clock.cc:648] HybridClock initialized: now 1786142351448785 us; error 26 us; skew 500 ppm
Aug 07 22:39:11 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142351, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:11.451454 20540 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:39:11.451921 20540 webserver.cc:460] Webserver started at http://127.28.82.194:43313/ using document root <none> and password file <none>
I20260807 22:39:11.452150 20540 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:11.452222 20540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:11.453536 20540 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:11.454190 20558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:11.454378 20540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:11.454504 20540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "97cf2f6f50784e1caf5dc8e96509adb7"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "c3faf3c608022bb6a1b3408c1420edcf"
server_key_iv: "9b6ea954af49521b2aa572ef0f7db370"
server_key_version: "encryptionkey@0"
I20260807 22:39:11.454895 20540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:11.463459 20540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:11.465523 20540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:11.465713 20540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:11.465955 20540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:11.466262 20540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:11.466320 20540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:11.466361 20540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:11.466390 20540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:11.472730 20540 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:34317
I20260807 22:39:11.472841 20735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:34317 every 8 connection(s)
I20260807 22:39:11.473198 20540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:39:11.474321 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:11.473940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:46769)
0807 22:39:11.474103 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.474177 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
W20260807 22:39:11.474782 20736 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:35113 (0 consecutive failures): Network error: Failed to ping master at 127.28.82.254:35113: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
I20260807 22:39:11.475162 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:11.475015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:37059)
0807 22:39:11.475089 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.475132 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 22:39:11.475442 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:11.475385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:37333)
0807 22:39:11.475407 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.475427 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260807 22:39:11.475556 20736 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:39:11.478956 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 20540
I20260807 22:39:11.479269 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:43617
--local_ip_for_outbound_sockets=127.28.82.195
--tserver_master_addrs=127.28.82.254:35113
--webserver_port=45173
--webserver_interface=127.28.82.195
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:11.564371 20739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:11.564577 20739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:11.564601 20739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:11.565742 20739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:11.565786 20739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:11.565812 20739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:39:11.567341 20739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:43617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=45173
--tserver_master_addrs=127.28.82.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:11.567576 20739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:11.567828 20739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:11.573029 20746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:11.573079 20745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:11.573091 20749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:11.573382 20739 server_base.cc:1034] running on GCE node
I20260807 22:39:11.573603 20739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:11.573877 20739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:11.575044 20739 hybrid_clock.cc:648] HybridClock initialized: now 1786142351575039 us; error 38 us; skew 500 ppm
Aug 07 22:39:11 dist-test-slave-7kzw krb5kdc[18598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142351, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:11.577673 20739 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:39:11.578114 20739 webserver.cc:460] Webserver started at http://127.28.82.195:45173/ using document root <none> and password file <none>
I20260807 22:39:11.578343 20739 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:11.578416 20739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:11.579849 20739 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:11.580521 20759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:11.580842 20739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:11.580968 20739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "08c64815edec4d6baf964973ee1dd0fc"
format_stamp: "Formatted at 2026-08-07 22:38:29 on dist-test-slave-7kzw"
server_key: "23cffd11a33667a01e7826de616acd61"
server_key_iv: "6fd871eedb31629eb2acd1155568da1b"
server_key_version: "encryptionkey@0"
I20260807 22:39:11.581313 20739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:11.588351 20739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:11.590319 20739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:11.590544 20739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:11.590775 20739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:11.591075 20739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:11.591135 20739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:11.591176 20739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:11.591202 20739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:11.597273 20739 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:43617
I20260807 22:39:11.597342 20928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:43617 every 8 connection(s)
I20260807 22:39:11.597724 20739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:39:11.598816 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:11.598329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:33931)
0807 22:39:11.598545 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.598660 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
W20260807 22:39:11.599207 20929 heartbeater.cc:644] Failed to heartbeat to 127.28.82.254:35113 (0 consecutive failures): Network error: Failed to ping master at 127.28.82.254:35113: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
I20260807 22:39:11.599503 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:11.599413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:47675)
0807 22:39:11.599460 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.599486 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:39:11.599787 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:11.599735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:47179)
0807 22:39:11.599755 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:11.599774 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260807 22:39:11.599908 20929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 22:39:11.605196 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 20739
I20260807 22:39:12.425959 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:12.425722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:53437)
0807 22:39:12.425852 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:12.425892 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260807 22:39:12.476176 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:12.475933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:54229)
0807 22:39:12.476079 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:12.476118 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
I20260807 22:39:12.600481 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:12.600242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:37159)
0807 22:39:12.600369 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:12.600412 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260807 22:39:12.793493 20940 raft_consensus.cc:482] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:12.793696 20940 raft_consensus.cc:504] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:39:12.794077 20940 leader_election.cc:288] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617), 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
Aug 07 22:39:12 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142351, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.194@KRBTEST.COM
Aug 07 22:39:12 dist-test-slave-7kzw krb5kdc[18598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142351, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.195@KRBTEST.COM
I20260807 22:39:12.805415 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:12.794562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:55221 (local address 127.28.82.194:34317)
0807 22:39:12.794711 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:12.794718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:12.794749 (+ 31us) server_negotiation.cc:407] Connection header received
0807 22:39:12.795351 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:12.795354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:12.795420 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:12.795570 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:12.796408 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:12.797279 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:12.798333 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:12.798813 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:12.798940 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:12.801526 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:12.801530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:12.801531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:12.801576 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:12.803513 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:12.804021 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:12.804025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:12.804025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:12.804141 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:12.804441 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:12.804442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:12.804443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:12.804700 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:12.804780 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:12.805141 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 22:39:12.805277 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260807 22:39:12.805454 20941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:12.794388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:34317 (local address 127.28.82.193:55221)
0807 22:39:12.794518 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:12.794531 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:12.794602 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:12.795709 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:12.795710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:12.796085 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:12.796282 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:12.796287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:12.797405 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:12.797406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:12.798231 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:12.798233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:12.798913 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:12.798914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:12.799041 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:12.799770 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:12.799775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:12.801397 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:12.803607 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:12.803615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:12.803617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:12.803941 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:12.804229 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:12.804230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:12.804230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:12.804373 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:12.804790 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:12.804798 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:12.804932 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 22:39:12.805080 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 22:39:12.805304 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":39,"thread_start_us":83,"threads_started":1}
I20260807 22:39:12.805488 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:12.794494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:51549 (local address 127.28.82.195:43617)
0807 22:39:12.794700 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:12.794713 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:12.794749 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:39:12.795351 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:12.795354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:12.795414 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:12.795580 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:12.796434 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:12.797308 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:12.798355 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:12.798837 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:12.798976 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:12.801907 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:12.801911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:12.801911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:12.801922 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:12.803660 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:12.804122 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:12.804125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:12.804125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:12.804280 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:12.804576 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:12.804577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:12.804578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:12.804918 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:12.805025 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:12.805231 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 22:39:12.805376 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":82,"threads_started":1}
I20260807 22:39:12.806069 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:12.794235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:43617 (local address 127.28.82.193:51549)
0807 22:39:12.794404 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:12.794419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:12.794531 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:12.795678 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:12.795681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:12.796090 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:12.796282 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:12.796287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:12.797441 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:12.797442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:12.798232 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:12.798235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:12.798945 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:12.798946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:12.799047 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:12.799763 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:12.799772 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:12.801791 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:12.803749 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:12.803750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:12.803751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:12.804009 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:12.804361 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:12.804362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:12.804362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:12.804449 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:12.805007 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:12.805008 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:12.805072 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 22:39:12.805146 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:39:12.805390 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":142}
W20260807 22:39:12.806336 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
W20260807 22:39:12.806784 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
I20260807 22:39:12.806831 20375 leader_election.cc:302] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0de04c931e9a48fba59cd81ddd8478e3; no voters: 08c64815edec4d6baf964973ee1dd0fc, 97cf2f6f50784e1caf5dc8e96509adb7
I20260807 22:39:12.806937 20940 raft_consensus.cc:2726] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:39:13.426863 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:13.426587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:45323)
0807 22:39:13.426762 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:13.426813 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260807 22:39:13.476961 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:13.476744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:40913)
0807 22:39:13.476877 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:13.476921 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260807 22:39:13.601294 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:13.601009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:33663)
0807 22:39:13.601204 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:13.601252 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":65,"threads_started":1}
I20260807 22:39:14.427682 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:14.427380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:47989)
0807 22:39:14.427542 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:14.427581 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
I20260807 22:39:14.477659 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:14.477389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:56225)
0807 22:39:14.477536 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:14.477580 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260807 22:39:14.602025 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:14.601782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:35033)
0807 22:39:14.601943 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:14.601982 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260807 22:39:14.994252 20955 raft_consensus.cc:482] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:14.994374 20955 raft_consensus.cc:504] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:39:14.994614 20955 leader_election.cc:288] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617), 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
W20260807 22:39:14.995103 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
W20260807 22:39:14.995199 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
I20260807 22:39:14.995236 20375 leader_election.cc:302] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0de04c931e9a48fba59cd81ddd8478e3; no voters: 08c64815edec4d6baf964973ee1dd0fc, 97cf2f6f50784e1caf5dc8e96509adb7
I20260807 22:39:14.995329 20955 raft_consensus.cc:2726] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:39:15.428391 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:15.428127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:55295)
0807 22:39:15.428300 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:15.428339 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260807 22:39:15.478432 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:15.478179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:40775)
0807 22:39:15.478331 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:15.478369 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260807 22:39:15.602818 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:15.602582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:52153)
0807 22:39:15.602732 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:15.602772 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":59,"threads_started":1}
I20260807 22:39:16.429136 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:16.428909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:58261)
0807 22:39:16.429049 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:16.429085 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260807 22:39:16.479264 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:16.479018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:44703)
0807 22:39:16.479176 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:16.479216 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
I20260807 22:39:16.603542 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:16.603316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:51777)
0807 22:39:16.603460 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:16.603498 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260807 22:39:17.429849 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:17.429612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:45983)
0807 22:39:17.429762 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:17.429797 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260807 22:39:17.480046 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:17.479803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:33359)
0807 22:39:17.479961 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:17.479998 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260807 22:39:17.604223 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:17.604022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:53173)
0807 22:39:17.604145 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:17.604181 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260807 22:39:17.787616 20968 raft_consensus.cc:482] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:17.787774 20968 raft_consensus.cc:504] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:39:17.787989 20968 leader_election.cc:288] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617), 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
W20260807 22:39:17.788383 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
W20260807 22:39:17.788444 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
I20260807 22:39:17.788458 20375 leader_election.cc:302] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0de04c931e9a48fba59cd81ddd8478e3; no voters: 08c64815edec4d6baf964973ee1dd0fc, 97cf2f6f50784e1caf5dc8e96509adb7
I20260807 22:39:17.788616 20968 raft_consensus.cc:2726] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:39:18.430672 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:18.430385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:59507)
0807 22:39:18.430575 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:18.430621 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":96,"threads_started":1}
I20260807 22:39:18.480746 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:18.480518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:53031)
0807 22:39:18.480661 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:18.480698 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260807 22:39:18.604918 20971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:18.604689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:51261)
0807 22:39:18.604833 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:18.604870 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260807 22:39:19.431344 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:19.431125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:57673)
0807 22:39:19.431254 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:19.431291 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260807 22:39:19.481472 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:19.481191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:59301)
0807 22:39:19.481368 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:19.481419 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
I20260807 22:39:19.605696 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:19.605471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:51753)
0807 22:39:19.605607 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:19.605647 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260807 22:39:20.432066 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:20.431794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:38649)
0807 22:39:20.431968 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:20.432008 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":68,"threads_started":1}
I20260807 22:39:20.482406 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:20.482155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:52471)
0807 22:39:20.482318 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:20.482358 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":66,"threads_started":1}
I20260807 22:39:20.606405 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:20.606205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:54635)
0807 22:39:20.606326 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:20.606363 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260807 22:39:21.432834 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:21.432574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:40641)
0807 22:39:21.432743 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:21.432782 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
I20260807 22:39:21.483156 20982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:21.482937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:37991)
0807 22:39:21.483072 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:21.483108 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260807 22:39:21.607115 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:21.606910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:53161)
0807 22:39:21.607031 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:21.607068 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260807 22:39:22.433560 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:22.433333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:49625)
0807 22:39:22.433469 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:22.433507 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260807 22:39:22.483964 20988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:22.483717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:50353)
0807 22:39:22.483873 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:22.483910 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260807 22:39:22.607885 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:22.607637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:51051)
0807 22:39:22.607801 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:22.607841 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260807 22:39:22.663673 20990 raft_consensus.cc:482] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:22.663776 20990 raft_consensus.cc:504] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:39:22.663982 20990 leader_election.cc:288] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617), 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
W20260807 22:39:22.664510 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
W20260807 22:39:22.664598 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
I20260807 22:39:22.664614 20375 leader_election.cc:302] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0de04c931e9a48fba59cd81ddd8478e3; no voters: 08c64815edec4d6baf964973ee1dd0fc, 97cf2f6f50784e1caf5dc8e96509adb7
I20260807 22:39:22.664695 20990 raft_consensus.cc:2726] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:39:23.434340 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:23.434089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:60397)
0807 22:39:23.434249 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:23.434287 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260807 22:39:23.484714 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:23.484404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:51947)
0807 22:39:23.484559 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:23.484664 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":62,"threads_started":1}
I20260807 22:39:23.608676 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:23.608451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:56039)
0807 22:39:23.608586 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:23.608624 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260807 22:39:24.357369 20995 raft_consensus.cc:482] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:24.357461 20995 raft_consensus.cc:504] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c64815edec4d6baf964973ee1dd0fc" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43617 } } peers { permanent_uuid: "97cf2f6f50784e1caf5dc8e96509adb7" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 34317 } } peers { permanent_uuid: "0de04c931e9a48fba59cd81ddd8478e3" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 45721 } }
I20260807 22:39:24.357609 20995 leader_election.cc:288] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617), 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317)
W20260807 22:39:24.358045 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97cf2f6f50784e1caf5dc8e96509adb7 (127.28.82.194:34317): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
W20260807 22:39:24.358177 20375 leader_election.cc:341] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08c64815edec4d6baf964973ee1dd0fc (127.28.82.195:43617): Not found: Tablet not found: 33e07b888a90418e8b681b39d3f6f7a1
I20260807 22:39:24.358219 20375 leader_election.cc:302] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0de04c931e9a48fba59cd81ddd8478e3; no voters: 08c64815edec4d6baf964973ee1dd0fc, 97cf2f6f50784e1caf5dc8e96509adb7
I20260807 22:39:24.358322 20995 raft_consensus.cc:2726] T 33e07b888a90418e8b681b39d3f6f7a1 P 0de04c931e9a48fba59cd81ddd8478e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 22:39:24.435158 20996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:24.434892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:38311)
0807 22:39:24.435067 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:24.435105 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260807 22:39:24.485427 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:24.485194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:59763)
0807 22:39:24.485333 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:24.485374 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260807 22:39:24.609467 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:24.609205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:59821)
0807 22:39:24.609381 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:24.609419 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260807 22:39:25.435910 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:25.435694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:42275)
0807 22:39:25.435822 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:25.435858 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260807 22:39:25.486200 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:25.485928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:45373)
0807 22:39:25.486103 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:25.486146 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260807 22:39:25.610240 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:25.609993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:52307)
0807 22:39:25.610152 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:25.610192 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260807 22:39:26.436766 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:26.436554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.193:52185)
0807 22:39:26.436682 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:26.436717 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260807 22:39:26.486991 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:26.486751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.194:60569)
0807 22:39:26.486905 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:26.486942 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":65,"threads_started":1}
I20260807 22:39:26.609001 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 20304
I20260807 22:39:26.613821 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:26.613481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:35113 (local address 127.28.82.195:42147)
0807 22:39:26.613729 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:26.613764 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.254:35113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":59,"threads_started":1}
I20260807 22:39:26.620993 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 20540
I20260807 22:39:26.630216 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 20739
2026-08-07T22:39:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58854 ms)
[----------] 30 tests from SecurityITest (246035 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:26 dist-test-slave-7kzw krb5kdc[21011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:26 dist-test-slave-7kzw krb5kdc[21011](info): set up 2 sockets
Aug 07 22:39:26 dist-test-slave-7kzw krb5kdc[21011](info): commencing operation
krb5kdc: starting...
W20260807 22:39:27.862598 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:27 dist-test-slave-7kzw krb5kdc[21011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:39:27.989056 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45703
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:45703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:28.072507 21030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:28.072751 21030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:28.072789 21030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:28.073942 21030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:28.074009 21030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:28.074033 21030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:28.074054 21030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:28.074082 21030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:28.075559 21030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:45703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:28.075830 21030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:28.076061 21030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:28.081228 21044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:28.081240 21038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:28.081240 21040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:28.081271 21030 server_base.cc:1034] running on GCE node
I20260807 22:39:28.081707 21030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:28.082041 21030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:28.083215 21030 hybrid_clock.cc:648] HybridClock initialized: now 1786142368083199 us; error 28 us; skew 500 ppm
Aug 07 22:39:28 dist-test-slave-7kzw krb5kdc[21011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142368, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:28.085731 21030 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:39:28.086180 21030 webserver.cc:460] Webserver started at http://127.28.82.254:39717/ using document root <none> and password file <none>
I20260807 22:39:28.086409 21030 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:28.086498 21030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:28.086659 21030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:28.088104 21030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "d6ea332b8af64fcf98f127fae1ef90d1"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "00fac7a5aba9523ffcac0bd37ac93cad"
server_key_iv: "4f8bd378945b06ae11e9b4954d63c4c0"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.088413 21030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "d6ea332b8af64fcf98f127fae1ef90d1"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "00fac7a5aba9523ffcac0bd37ac93cad"
server_key_iv: "4f8bd378945b06ae11e9b4954d63c4c0"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.089672 21030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:28.090356 21052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.090559 21030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:39:28.090674 21030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "d6ea332b8af64fcf98f127fae1ef90d1"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "00fac7a5aba9523ffcac0bd37ac93cad"
server_key_iv: "4f8bd378945b06ae11e9b4954d63c4c0"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.090765 21030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:28.102988 21030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:28.104807 21030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:28.104967 21030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:28.108981 21030 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:45703
I20260807 22:39:28.109016 21127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:45703 every 8 connection(s)
I20260807 22:39:28.109360 21030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:28.110196 21128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:28.112428 21128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1: Bootstrap starting.
I20260807 22:39:28.112972 21128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:28.113206 21128 log.cc:824] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:28.113904 21128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1: No bootstrap required, opened a new log
I20260807 22:39:28.114295 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 21030
I20260807 22:39:28.114421 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:28.114704 29003 external_mini_cluster.cc:1428] Setting key 2ad0ed8f81837815d68621f950e31687
I20260807 22:39:28.116442 21128 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea332b8af64fcf98f127fae1ef90d1" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45703 } }
I20260807 22:39:28.116624 21128 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:28.116670 21128 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ea332b8af64fcf98f127fae1ef90d1, State: Initialized, Role: FOLLOWER
I20260807 22:39:28.116815 21128 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea332b8af64fcf98f127fae1ef90d1" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45703 } }
I20260807 22:39:28.116907 21128 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:28.116941 21128 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:28.116994 21128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:28.117636 21128 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea332b8af64fcf98f127fae1ef90d1" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45703 } }
I20260807 22:39:28.117727 21128 leader_election.cc:302] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6ea332b8af64fcf98f127fae1ef90d1; no voters:
I20260807 22:39:28.117959 21128 leader_election.cc:288] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:28.117990 21135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:28.118108 21135 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [term 1 LEADER]: Becoming Leader. State: Replica: d6ea332b8af64fcf98f127fae1ef90d1, State: Running, Role: LEADER
I20260807 22:39:28.118227 21128 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:28.118263 21135 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea332b8af64fcf98f127fae1ef90d1" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45703 } }
I20260807 22:39:28.118660 21135 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6ea332b8af64fcf98f127fae1ef90d1. Latest consensus state: current_term: 1 leader_uuid: "d6ea332b8af64fcf98f127fae1ef90d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea332b8af64fcf98f127fae1ef90d1" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45703 } } }
I20260807 22:39:28.118674 21136 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6ea332b8af64fcf98f127fae1ef90d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea332b8af64fcf98f127fae1ef90d1" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45703 } } }
I20260807 22:39:28.118848 21135 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:28.118860 21136 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:28.119174 21148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:28.119750 21148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:28.121421 21148 catalog_manager.cc:1347] Generated new cluster ID: 1b310c9352bc4ec48f74c3886937728e
I20260807 22:39:28.121477 21148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:39:28 dist-test-slave-7kzw krb5kdc[21011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:28.126312 21134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.115260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.28.82.254:45703)
0807 22:39:28.115429 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.115437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.115462 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:39:28.116377 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.116384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.116431 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.116544 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:28.117370 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.118575 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.119715 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.120330 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.120454 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.123021 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:28.123025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:28.123026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:28.123038 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:28.124724 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.125133 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.125137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.125137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.125232 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.125448 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.125449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.125449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.125682 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:28.125857 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.125954 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.126106 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260807 22:39:28.140954 21148 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:28.141644 21148 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:28.148726 21148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1: Generated new TSK 0
I20260807 22:39:28.148919 21148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:39:28.185887 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45703
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:28.269268 21169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:28.269469 21169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:28.269493 21169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:28.270709 21169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:28.270756 21169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:28.270782 21169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:39:28.272233 21169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:28.272450 21169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:28.272666 21169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:28.277797 21178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:28.277808 21174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:28.277848 21169 server_base.cc:1034] running on GCE node
W20260807 22:39:28.277808 21175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:28.278256 21169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:28.278528 21169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:28.279678 21169 hybrid_clock.cc:648] HybridClock initialized: now 1786142368279657 us; error 39 us; skew 500 ppm
Aug 07 22:39:28 dist-test-slave-7kzw krb5kdc[21011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142368, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:28.282181 21169 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:39:28.282675 21169 webserver.cc:460] Webserver started at http://127.28.82.193:43503/ using document root <none> and password file <none>
I20260807 22:39:28.282910 21169 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:28.282958 21169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:28.283124 21169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:28.284181 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "1c184bc93b9b44d1b778a69be326acc6"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "0514ba7933c9ca99a21b36aeb14c7a5c"
server_key_iv: "c19aa6202597029f9f924be036803078"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.284543 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "1c184bc93b9b44d1b778a69be326acc6"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "0514ba7933c9ca99a21b36aeb14c7a5c"
server_key_iv: "c19aa6202597029f9f924be036803078"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.285849 21169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:39:28.286633 21187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.286828 21169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:39:28.286923 21169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "1c184bc93b9b44d1b778a69be326acc6"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "0514ba7933c9ca99a21b36aeb14c7a5c"
server_key_iv: "c19aa6202597029f9f924be036803078"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.286999 21169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:28.324270 21169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:28.326382 21169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:28.326581 21169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:28.326826 21169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:28.327157 21169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:28.327214 21169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.327265 21169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:28.327306 21169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.333513 21169 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:39593
I20260807 22:39:28.333567 21345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:39593 every 8 connection(s)
I20260807 22:39:28.333868 21169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:39:28 dist-test-slave-7kzw krb5kdc[21011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142368, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:28.341540 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 21169
I20260807 22:39:28.341626 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:39:28.341840 29003 external_mini_cluster.cc:1428] Setting key 2f3e905319e3e0b388311c849b665076
I20260807 22:39:28.343137 21134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.334663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:59789 (local address 127.28.82.254:45703)
0807 22:39:28.334732 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.334734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.334943 (+ 209us) server_negotiation.cc:407] Connection header received
0807 22:39:28.335423 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.335424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.335434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.335496 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:28.336396 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.336923 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.337846 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.338043 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.338110 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.340131 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:28.340132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:28.340133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:28.340139 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:28.341616 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.342100 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.342101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.342102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.342167 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.342446 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.342447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.342447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.342593 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:28.342674 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.342947 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.343034 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:39:28.343313 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.334639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45703 (local address 127.28.82.193:59789)
0807 22:39:28.334812 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.334853 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:28.335339 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:28.335547 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:28.335551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:28.336108 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:28.336303 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.336307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.337006 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.337007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.337767 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.337768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.338139 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.338140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.338252 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.338678 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:28.338686 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:28.340044 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:28.341696 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:28.341704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:28.341706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:28.342004 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:28.342245 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:28.342246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:28.342247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:28.342373 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:28.342665 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:28.342673 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:28.342811 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 22:39:28.342923 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 22:39:28.343073 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260807 22:39:28.343901 21346 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45703
I20260807 22:39:28.343994 21346 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:28.344201 21346 heartbeater.cc:511] Master 127.28.82.254:45703 requested a full tablet report, sending...
I20260807 22:39:28.344755 21076 ts_manager.cc:194] Registered new tserver with Master: 1c184bc93b9b44d1b778a69be326acc6 (127.28.82.193:39593)
I20260807 22:39:28.345359 21076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:59789
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:39:28.368784 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45703
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:28.450407 21352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:28.450668 21352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:28.450709 21352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:28.451843 21352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:28.451915 21352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:28.451960 21352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:39:28.453485 21352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:28.453756 21352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:28.453986 21352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:28.459033 21366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:28.459070 21368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:28.459059 21362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:28.459296 21352 server_base.cc:1034] running on GCE node
I20260807 22:39:28.459530 21352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:28.459816 21352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:28.460986 21352 hybrid_clock.cc:648] HybridClock initialized: now 1786142368460964 us; error 41 us; skew 500 ppm
Aug 07 22:39:28 dist-test-slave-7kzw krb5kdc[21011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142368, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:28.463567 21352 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:39:28.463994 21352 webserver.cc:460] Webserver started at http://127.28.82.194:35653/ using document root <none> and password file <none>
I20260807 22:39:28.464212 21352 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:28.464284 21352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:28.464437 21352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:28.465456 21352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "d8e8eff99b584ba19ca293b7003cd68d"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "d2d437c38b2f70e3e8fa1822acd2c8f4"
server_key_iv: "dc975e5666e8c3c5830bf87d098314cd"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.465826 21352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "d8e8eff99b584ba19ca293b7003cd68d"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "d2d437c38b2f70e3e8fa1822acd2c8f4"
server_key_iv: "dc975e5666e8c3c5830bf87d098314cd"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.467345 21352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:28.468118 21376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.468313 21352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:28.468415 21352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "d8e8eff99b584ba19ca293b7003cd68d"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "d2d437c38b2f70e3e8fa1822acd2c8f4"
server_key_iv: "dc975e5666e8c3c5830bf87d098314cd"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.468497 21352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:28.478039 21352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:28.480321 21352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:28.480481 21352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:28.480748 21352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:28.481045 21352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:28.481099 21352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.481140 21352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:28.481165 21352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.487481 21352 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:41693
I20260807 22:39:28.487532 21537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:41693 every 8 connection(s)
I20260807 22:39:28.487839 21352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:39:28.493587 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 21352
I20260807 22:39:28.493708 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:39:28.493959 29003 external_mini_cluster.cc:1428] Setting key f8fe1de9a1055ac9c2d0320886f8e2de
Aug 07 22:39:28 dist-test-slave-7kzw krb5kdc[21011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142368, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:28.498003 21134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.488532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:55501 (local address 127.28.82.254:45703)
0807 22:39:28.488594 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.488596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.488794 (+ 198us) server_negotiation.cc:407] Connection header received
0807 22:39:28.489332 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.489333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.489342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.489398 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:28.490412 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.491022 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.492014 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.492194 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.492251 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.494821 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:28.494822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:28.494823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:28.494830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:28.496440 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.496992 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.496993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.496994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.497050 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.497349 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.497350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.497350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.497449 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:28.497497 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.497875 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.497937 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:39:28.498317 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.488497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45703 (local address 127.28.82.194:55501)
0807 22:39:28.488665 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.488706 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:28.489248 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:28.489447 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:28.489450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:28.490127 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:28.490332 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.490338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.491128 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.491129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.491935 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.491938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.492270 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.492271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.492379 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.493021 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:28.493030 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:28.494735 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:28.496524 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:28.496531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:28.496533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:28.496904 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:28.497121 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:28.497122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:28.497122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:28.497275 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:28.497512 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:28.497518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:28.497729 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 22:39:28.497848 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 22:39:28.498037 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260807 22:39:28.498896 21538 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45703
I20260807 22:39:28.499004 21538 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:28.499197 21538 heartbeater.cc:511] Master 127.28.82.254:45703 requested a full tablet report, sending...
I20260807 22:39:28.499557 21076 ts_manager.cc:194] Registered new tserver with Master: d8e8eff99b584ba19ca293b7003cd68d (127.28.82.194:41693)
I20260807 22:39:28.499935 21076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:55501
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:39:28.521957 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45703
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:28.605187 21547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:28.605384 21547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:28.605409 21547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:28.606801 21547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:28.606844 21547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:28.606873 21547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:39:28.608372 21547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:28.608608 21547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:28.608870 21547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:28.614108 21557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:28.614109 21559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:28.614133 21555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:28.614423 21547 server_base.cc:1034] running on GCE node
I20260807 22:39:28.614670 21547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:28.614988 21547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:28.616170 21547 hybrid_clock.cc:648] HybridClock initialized: now 1786142368616142 us; error 51 us; skew 500 ppm
Aug 07 22:39:28 dist-test-slave-7kzw krb5kdc[21011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142368, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:28.619032 21547 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:39:28.619557 21547 webserver.cc:460] Webserver started at http://127.28.82.195:45607/ using document root <none> and password file <none>
I20260807 22:39:28.619823 21547 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:28.619895 21547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:28.620059 21547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:28.621111 21547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "95987fb5232d4351941ffd23d3f9ad95"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "d08314da3c1b4d70fea8eaf646b7a9a4"
server_key_iv: "10997bd0ed3916e52614430743ab1451"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.621486 21547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "95987fb5232d4351941ffd23d3f9ad95"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "d08314da3c1b4d70fea8eaf646b7a9a4"
server_key_iv: "10997bd0ed3916e52614430743ab1451"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.622900 21547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:28.623654 21566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.623893 21547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:28.624022 21547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "95987fb5232d4351941ffd23d3f9ad95"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "d08314da3c1b4d70fea8eaf646b7a9a4"
server_key_iv: "10997bd0ed3916e52614430743ab1451"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.624099 21547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:28.636058 21547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:28.638404 21547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:28.638628 21547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:28.638872 21547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:28.639196 21547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:28.639249 21547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.639284 21547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:28.639312 21547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.645516 21547 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:36643
I20260807 22:39:28.645557 21741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:36643 every 8 connection(s)
I20260807 22:39:28.645942 21547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:39:28.647054 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 21547
I20260807 22:39:28.647173 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:39:28.647367 29003 external_mini_cluster.cc:1428] Setting key faa93ef01631675ad482c0dc6c9d838e
Aug 07 22:39:28 dist-test-slave-7kzw krb5kdc[21011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142368, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:28.655686 21134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.646787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:58791 (local address 127.28.82.254:45703)
0807 22:39:28.646873 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.646876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.647094 (+ 218us) server_negotiation.cc:407] Connection header received
0807 22:39:28.647882 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.647883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.647893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.647963 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:28.648962 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.649608 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.650639 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.650825 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.650883 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.653048 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:28.653049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:28.653050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:28.653057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:28.654414 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.654850 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.654851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.654851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.654907 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.655123 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.655123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.655123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.655227 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:28.655277 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.655519 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.655603 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 22:39:28.655815 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.646725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45703 (local address 127.28.82.195:58791)
0807 22:39:28.646931 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.646982 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:28.647716 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:28.648035 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:28.648040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:28.648697 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:28.648885 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.648889 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.649703 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.649704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.650547 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.650550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.650902 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.650902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.651003 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.651519 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:28.651527 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:28.652937 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:28.654542 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:28.654549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:28.654551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:28.654785 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:28.654968 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:28.654969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:28.654969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:28.655069 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:28.655278 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:28.655283 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:28.655391 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 22:39:28.655470 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:39:28.655587 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
I20260807 22:39:28.656315 21744 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45703
I20260807 22:39:28.656419 21744 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:28.656618 21744 heartbeater.cc:511] Master 127.28.82.254:45703 requested a full tablet report, sending...
I20260807 22:39:28.657034 21076 ts_manager.cc:194] Registered new tserver with Master: 95987fb5232d4351941ffd23d3f9ad95 (127.28.82.195:36643)
I20260807 22:39:28.657377 21076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:58791
I20260807 22:39:28.659166 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:39:28.665984 21134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.660353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40448 (local address 127.28.82.254:45703)
0807 22:39:28.660419 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.660421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.660512 (+ 91us) server_negotiation.cc:407] Connection header received
0807 22:39:28.660547 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.660548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.660556 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.660607 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:28.661178 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.661645 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.662309 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.662531 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.662587 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.663694 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:28.663696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:28.663696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:28.663702 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:28.664962 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.665270 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.665270 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.665271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.665322 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:28.665493 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:28.665493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:28.665494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:28.665667 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:28.665743 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.665823 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.665882 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 22:39:28.667640 21076 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:39:28.669225 21076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:39:28.676087 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.671886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.28.82.195:36643)
0807 22:39:28.672087 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.672092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.672319 (+ 227us) server_negotiation.cc:407] Connection header received
0807 22:39:28.672415 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.672417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.672434 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.672509 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:28.672710 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.673511 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.674958 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.675609 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.675735 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.675798 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.675963 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.676003 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":89,"threads_started":1}
I20260807 22:39:28.676076 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.671736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:36643 (local address 127.0.0.1:59444)
0807 22:39:28.672241 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.672251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:28.672318 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:28.672535 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:28.672538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:28.672540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:28.672617 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.672619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.673630 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.673632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.674869 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.674872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.675738 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.675740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.675834 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.675848 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:39:28.675945 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 22:39:28.676003 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":5,"thread_start_us":78,"threads_started":1}
I20260807 22:39:28.676400 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.671756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:41693 (local address 127.0.0.1:36918)
0807 22:39:28.672404 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.672414 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:28.672457 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:28.672811 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:28.672812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:28.672814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:28.672857 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.672858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.674118 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.674119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.675249 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.675252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.676174 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.676174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.676226 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.676229 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:28.676276 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 22:39:28.676315 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":87,"threads_started":1}
I20260807 22:39:28.676671 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.671735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:39593 (local address 127.0.0.1:51506)
0807 22:39:28.671902 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.671957 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:28.672025 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:28.672581 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:28.672582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:28.672584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:28.672644 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.672646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.673985 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.673986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.675108 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.675111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.676110 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.676111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.676349 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.676352 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:28.676410 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 22:39:28.676450 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":67,"thread_start_us":65,"threads_started":1}
I20260807 22:39:28.676666 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.672007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36918 (local address 127.28.82.194:41693)
0807 22:39:28.672230 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.672235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.672515 (+ 280us) server_negotiation.cc:407] Connection header received
0807 22:39:28.672634 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.672636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.672653 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.672726 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:28.672944 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.674004 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.675338 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.676031 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.676094 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.676143 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.676522 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.676570 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":87,"threads_started":1}
I20260807 22:39:28.676656 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.672017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51506 (local address 127.28.82.193:39593)
0807 22:39:28.672207 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.672211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.672222 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:39:28.672279 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.672281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.672299 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.672390 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:28.672735 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.673872 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.675193 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.675925 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.676000 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.676058 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.676441 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.676576 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
I20260807 22:39:28.676923 21646 tablet_service.cc:1467] Processing CreateTablet for tablet f19adbc038e34f8c8403a6de1da46701 (DEFAULT_TABLE table=test-table [id=63380b3f718b43828845f3b711a2f980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:28.677318 21646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19adbc038e34f8c8403a6de1da46701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:28.677285 21443 tablet_service.cc:1467] Processing CreateTablet for tablet f19adbc038e34f8c8403a6de1da46701 (DEFAULT_TABLE table=test-table [id=63380b3f718b43828845f3b711a2f980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:28.677628 21443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19adbc038e34f8c8403a6de1da46701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:28.677718 21261 tablet_service.cc:1467] Processing CreateTablet for tablet f19adbc038e34f8c8403a6de1da46701 (DEFAULT_TABLE table=test-table [id=63380b3f718b43828845f3b711a2f980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:28.678021 21261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19adbc038e34f8c8403a6de1da46701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:28.680433 21769 tablet_bootstrap.cc:492] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Bootstrap starting.
I20260807 22:39:28.681173 21769 tablet_bootstrap.cc:654] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:28.681296 21771 tablet_bootstrap.cc:492] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6: Bootstrap starting.
I20260807 22:39:28.681612 21769 log.cc:824] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:28.681720 21770 tablet_bootstrap.cc:492] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: Bootstrap starting.
I20260807 22:39:28.681969 21771 tablet_bootstrap.cc:654] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:28.682366 21769 tablet_bootstrap.cc:492] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: No bootstrap required, opened a new log
I20260807 22:39:28.682396 21770 tablet_bootstrap.cc:654] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:28.682456 21769 ts_tablet_manager.cc:1397] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:39:28.682641 21771 log.cc:824] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:28.682962 21770 log.cc:824] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:28.683540 21771 tablet_bootstrap.cc:492] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6: No bootstrap required, opened a new log
I20260807 22:39:28.683619 21771 ts_tablet_manager.cc:1397] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:39:28.683842 21770 tablet_bootstrap.cc:492] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: No bootstrap required, opened a new log
I20260807 22:39:28.683954 21770 ts_tablet_manager.cc:1397] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:39:28.684854 21769 raft_consensus.cc:348] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.684934 21771 raft_consensus.cc:348] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.685051 21769 raft_consensus.cc:374] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:28.685081 21771 raft_consensus.cc:374] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:28.685102 21769 raft_consensus.cc:729] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e8eff99b584ba19ca293b7003cd68d, State: Initialized, Role: FOLLOWER
I20260807 22:39:28.685128 21771 raft_consensus.cc:729] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c184bc93b9b44d1b778a69be326acc6, State: Initialized, Role: FOLLOWER
I20260807 22:39:28.685233 21771 consensus_queue.cc:260] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.685262 21769 consensus_queue.cc:260] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.685520 21771 ts_tablet_manager.cc:1428] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:39:28.685542 21346 heartbeater.cc:503] Master 127.28.82.254:45703 was elected leader, sending a full tablet report...
I20260807 22:39:28.685541 21769 ts_tablet_manager.cc:1428] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:39:28.685681 21538 heartbeater.cc:503] Master 127.28.82.254:45703 was elected leader, sending a full tablet report...
I20260807 22:39:28.686095 21770 raft_consensus.cc:348] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.686254 21770 raft_consensus.cc:374] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:28.686301 21770 raft_consensus.cc:729] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95987fb5232d4351941ffd23d3f9ad95, State: Initialized, Role: FOLLOWER
I20260807 22:39:28.686416 21770 consensus_queue.cc:260] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.686775 21744 heartbeater.cc:503] Master 127.28.82.254:45703 was elected leader, sending a full tablet report...
I20260807 22:39:28.686797 21770 ts_tablet_manager.cc:1428] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260807 22:39:28.738438 21539 tablet.cc:2369] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:39:28.746222 21776 raft_consensus.cc:482] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:28.746335 21776 raft_consensus.cc:504] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.746707 21776 leader_election.cc:288] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95987fb5232d4351941ffd23d3f9ad95 (127.28.82.195:36643), 1c184bc93b9b44d1b778a69be326acc6 (127.28.82.193:39593)
I20260807 22:39:28.749563 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.746905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:36643 (local address 127.28.82.194:52001)
0807 22:39:28.746978 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.746993 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:28.747050 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:28.747216 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:28.747217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:28.747219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:28.747304 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.747306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.747964 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.747965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.748703 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.748705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.749295 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.749296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.749378 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.749381 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:28.749451 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:39:28.749492 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 22:39:28.749563 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.747012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:52001 (local address 127.28.82.195:36643)
0807 22:39:28.747068 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.747071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.747080 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:39:28.747116 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.747117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.747127 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.747212 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:28.747378 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.747877 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.748824 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.749209 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.749295 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.749304 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.749456 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.749499 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 22:39:28.749742 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.746905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:39593 (local address 127.28.82.194:47337)
0807 22:39:28.747129 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.747138 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:28.747223 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:28.747360 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:28.747360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:28.747362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:28.747442 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.747444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.748068 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.748069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.748883 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:28.748886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.749466 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.749468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:28.749539 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.749543 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:28.749624 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 22:39:28.749661 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":52,"thread_start_us":111,"threads_started":1}
I20260807 22:39:28.749734 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.747033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:47337 (local address 127.28.82.193:39593)
0807 22:39:28.747083 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.747085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.747207 (+ 122us) server_negotiation.cc:407] Connection header received
0807 22:39:28.747290 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.747291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.747302 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.747348 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:28.747521 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.747982 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.748980 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.749368 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.749458 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.749466 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.749633 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.749672 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 22:39:28.749931 21671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19adbc038e34f8c8403a6de1da46701" candidate_uuid: "d8e8eff99b584ba19ca293b7003cd68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95987fb5232d4351941ffd23d3f9ad95" is_pre_election: true
I20260807 22:39:28.750028 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19adbc038e34f8c8403a6de1da46701" candidate_uuid: "d8e8eff99b584ba19ca293b7003cd68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c184bc93b9b44d1b778a69be326acc6" is_pre_election: true
I20260807 22:39:28.750120 21671 raft_consensus.cc:2454] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e8eff99b584ba19ca293b7003cd68d in term 0.
I20260807 22:39:28.750185 21283 raft_consensus.cc:2454] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e8eff99b584ba19ca293b7003cd68d in term 0.
I20260807 22:39:28.750396 21378 leader_election.cc:302] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95987fb5232d4351941ffd23d3f9ad95, d8e8eff99b584ba19ca293b7003cd68d; no voters:
I20260807 22:39:28.750562 21776 raft_consensus.cc:2781] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:39:28.750643 21776 raft_consensus.cc:482] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:39:28.750665 21776 raft_consensus.cc:3037] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:28.751334 21776 raft_consensus.cc:504] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.751477 21776 leader_election.cc:288] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [CANDIDATE]: Term 1 election: Requested vote from peers 95987fb5232d4351941ffd23d3f9ad95 (127.28.82.195:36643), 1c184bc93b9b44d1b778a69be326acc6 (127.28.82.193:39593)
I20260807 22:39:28.751659 21671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19adbc038e34f8c8403a6de1da46701" candidate_uuid: "d8e8eff99b584ba19ca293b7003cd68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95987fb5232d4351941ffd23d3f9ad95"
I20260807 22:39:28.751705 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19adbc038e34f8c8403a6de1da46701" candidate_uuid: "d8e8eff99b584ba19ca293b7003cd68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c184bc93b9b44d1b778a69be326acc6"
I20260807 22:39:28.751744 21671 raft_consensus.cc:3037] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:28.751791 21283 raft_consensus.cc:3037] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:28.752549 21671 raft_consensus.cc:2454] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8e8eff99b584ba19ca293b7003cd68d in term 1.
I20260807 22:39:28.752560 21283 raft_consensus.cc:2454] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8e8eff99b584ba19ca293b7003cd68d in term 1.
I20260807 22:39:28.752767 21382 leader_election.cc:302] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c184bc93b9b44d1b778a69be326acc6, d8e8eff99b584ba19ca293b7003cd68d; no voters:
I20260807 22:39:28.752862 21776 raft_consensus.cc:2781] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:28.753054 21776 raft_consensus.cc:686] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 1 LEADER]: Becoming Leader. State: Replica: d8e8eff99b584ba19ca293b7003cd68d, State: Running, Role: LEADER
I20260807 22:39:28.753158 21776 consensus_queue.cc:237] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } }
I20260807 22:39:28.753998 21075 catalog_manager.cc:5495] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d reported cstate change: term changed from 0 to 1, leader changed from <none> to d8e8eff99b584ba19ca293b7003cd68d (127.28.82.194). New cstate: current_term: 1 leader_uuid: "d8e8eff99b584ba19ca293b7003cd68d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e8eff99b584ba19ca293b7003cd68d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 } health_report { overall_health: UNKNOWN } } }
I20260807 22:39:28.768285 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.765841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36928 (local address 127.28.82.194:41693)
0807 22:39:28.765899 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.765901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.765910 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:39:28.765952 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.765953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.765969 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.766036 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:39:28.766171 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.766626 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.767497 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.767691 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.767749 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.767930 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:39:28.768025 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:39:28.768086 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.768197 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.768215 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 22:39:28.769958 21283 raft_consensus.cc:1261] T f19adbc038e34f8c8403a6de1da46701 P 1c184bc93b9b44d1b778a69be326acc6 [term 1 FOLLOWER]: Refusing update from remote peer d8e8eff99b584ba19ca293b7003cd68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:28.769979 21671 raft_consensus.cc:1261] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 1 FOLLOWER]: Refusing update from remote peer d8e8eff99b584ba19ca293b7003cd68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:28.770314 21782 consensus_queue.cc:1035] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "95987fb5232d4351941ffd23d3f9ad95" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:28.770452 21776 consensus_queue.cc:1035] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c184bc93b9b44d1b778a69be326acc6" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 39593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:28.773255 21788 mvcc.cc:205] Tried to move back new op lower bound from 7316039142479282176 to 7316039142413504512. Current Snapshot: MvccSnapshot[applied={T|T < 7316039142479282176}]
I20260807 22:39:28.775315 21075 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:28.775419 21075 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:28.776142 21075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6ea332b8af64fcf98f127fae1ef90d1: Sending DeleteTablet for 3 replicas of tablet f19adbc038e34f8c8403a6de1da46701
I20260807 22:39:28.776466 21443 tablet_service.cc:1514] Processing DeleteTablet for tablet f19adbc038e34f8c8403a6de1da46701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:28 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:36918
I20260807 22:39:28.776510 21261 tablet_service.cc:1514] Processing DeleteTablet for tablet f19adbc038e34f8c8403a6de1da46701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:28 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:51506
I20260807 22:39:28.776775 21646 tablet_service.cc:1514] Processing DeleteTablet for tablet f19adbc038e34f8c8403a6de1da46701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:28 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:59444
I20260807 22:39:28.777026 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 21169
I20260807 22:39:28.788180 21802 tablet_replica.cc:330] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: stopping tablet replica
I20260807 22:39:28.788324 21802 raft_consensus.cc:2229] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:39:28.788436 21802 raft_consensus.cc:2258] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:28.789569 21802 ts_tablet_manager.cc:1905] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:28.791358 21802 ts_tablet_manager.cc:1918] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:28.791445 21802 log.cc:1196] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/f19adbc038e34f8c8403a6de1da46701
I20260807 22:39:28.791731 21802 ts_tablet_manager.cc:1939] T f19adbc038e34f8c8403a6de1da46701 P d8e8eff99b584ba19ca293b7003cd68d: Deleting consensus metadata
I20260807 22:39:28.792172 21058 catalog_manager.cc:4841] TS d8e8eff99b584ba19ca293b7003cd68d (127.28.82.194:41693): tablet f19adbc038e34f8c8403a6de1da46701 (table test-table [id=63380b3f718b43828845f3b711a2f980]) successfully deleted
I20260807 22:39:28.799046 21804 tablet_replica.cc:330] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: stopping tablet replica
I20260807 22:39:28.799284 21804 raft_consensus.cc:2229] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:28.799463 21804 raft_consensus.cc:2258] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:28.800370 21804 ts_tablet_manager.cc:1905] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:28.801757 21804 ts_tablet_manager.cc:1918] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:28.801857 21804 log.cc:1196] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/f19adbc038e34f8c8403a6de1da46701
I20260807 22:39:28.802150 21804 ts_tablet_manager.cc:1939] T f19adbc038e34f8c8403a6de1da46701 P 95987fb5232d4351941ffd23d3f9ad95: Deleting consensus metadata
I20260807 22:39:28.802639 21056 catalog_manager.cc:4841] TS 95987fb5232d4351941ffd23d3f9ad95 (127.28.82.195:36643): tablet f19adbc038e34f8c8403a6de1da46701 (table test-table [id=63380b3f718b43828845f3b711a2f980]) successfully deleted
W20260807 22:39:28.807412 21059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:39593 (error 108)
I20260807 22:39:28.808043 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.807896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:39593 (local address 127.0.0.1:51516)
0807 22:39:28.807956 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:28.808000 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 22:39:28.808061 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 21352
W20260807 22:39:28.814653 21059 catalog_manager.cc:4584] TS 1c184bc93b9b44d1b778a69be326acc6 (127.28.82.193:39593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f19adbc038e34f8c8403a6de1da46701: Network error: Client connection negotiation failed: client connection to 127.28.82.193:39593: connect: Connection refused (error 111)
I20260807 22:39:28.816030 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 21547
I20260807 22:39:28.823864 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 21030
2026-08-07T22:39:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2202 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T22:39:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:28Z Disabled control of system clock
I20260807 22:39:28.864327 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:34921
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:34921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:28.945972 21811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:28.946182 21811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:28.946202 21811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:28.947373 21811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:28.947418 21811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:28.947431 21811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:28.947443 21811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:28.947456 21811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:28.948729 21811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:34921
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:34921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:28.948922 21811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:28.949155 21811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:28.954159 21817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:28.954149 21824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:28.954218 21811 server_base.cc:1034] running on GCE node
W20260807 22:39:28.954149 21819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:28.954725 21811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:28.955029 21811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:28.956223 21811 hybrid_clock.cc:648] HybridClock initialized: now 1786142368956202 us; error 44 us; skew 500 ppm
I20260807 22:39:28.957500 21811 webserver.cc:460] Webserver started at http://127.28.82.254:34713/ using document root <none> and password file <none>
I20260807 22:39:28.957739 21811 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:28.957782 21811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:28.957875 21811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:28.958941 21811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "bdf0ea46a3a641e3bf6400410c0294c5"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "606fe035d391ab9d0e5c06c4f29b4c77"
server_key_iv: "b79f36fe6e2b91c7eaaab2a0fcb45ed2"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.959255 21811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "bdf0ea46a3a641e3bf6400410c0294c5"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "606fe035d391ab9d0e5c06c4f29b4c77"
server_key_iv: "b79f36fe6e2b91c7eaaab2a0fcb45ed2"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.960506 21811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:39:28.961299 21830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:28.961508 21811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:28.961622 21811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "bdf0ea46a3a641e3bf6400410c0294c5"
format_stamp: "Formatted at 2026-08-07 22:39:28 on dist-test-slave-7kzw"
server_key: "606fe035d391ab9d0e5c06c4f29b4c77"
server_key_iv: "b79f36fe6e2b91c7eaaab2a0fcb45ed2"
server_key_version: "encryptionkey@0"
I20260807 22:39:28.961723 21811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:28.971901 21811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:28.972196 21811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:28.972354 21811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:28.976506 21811 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:34921
I20260807 22:39:28.976543 21908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:34921 every 8 connection(s)
I20260807 22:39:28.976881 21811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:28.977857 21909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:28.978945 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 21811
I20260807 22:39:28.979065 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:28.979283 29003 external_mini_cluster.cc:1428] Setting key 4a45ca1ff9bb81b724762ceed8b1665d
I20260807 22:39:28.980783 21909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5: Bootstrap starting.
I20260807 22:39:28.981448 21909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:28.981739 21909 log.cc:824] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:28.982594 21909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5: No bootstrap required, opened a new log
I20260807 22:39:28.985137 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:28.979843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.28.82.254:34921)
0807 22:39:28.980050 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:28.980057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:28.980089 (+ 32us) server_negotiation.cc:407] Connection header received
0807 22:39:28.980597 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:28.980604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:28.980672 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:28.980786 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:28.981021 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.981992 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.982908 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:28.983535 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:28.983664 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:28.984341 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:28.984346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:28.984346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:28.984566 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:28.984614 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:28.984633 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:28.984717 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:28.984780 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 22:39:28.984874 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260807 22:39:28.985272 21909 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0ea46a3a641e3bf6400410c0294c5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 34921 } }
I20260807 22:39:28.985441 21909 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:28.985491 21909 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf0ea46a3a641e3bf6400410c0294c5, State: Initialized, Role: FOLLOWER
I20260807 22:39:28.985628 21909 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0ea46a3a641e3bf6400410c0294c5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 34921 } }
I20260807 22:39:28.985713 21909 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:28.985757 21909 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:28.985808 21909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:28.986370 21909 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0ea46a3a641e3bf6400410c0294c5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 34921 } }
I20260807 22:39:28.986532 21909 leader_election.cc:302] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdf0ea46a3a641e3bf6400410c0294c5; no voters:
I20260807 22:39:28.986744 21909 leader_election.cc:288] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:28.986795 21917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:28.986997 21917 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [term 1 LEADER]: Becoming Leader. State: Replica: bdf0ea46a3a641e3bf6400410c0294c5, State: Running, Role: LEADER
I20260807 22:39:28.987078 21909 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:28.987121 21917 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0ea46a3a641e3bf6400410c0294c5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 34921 } }
I20260807 22:39:28.987524 21918 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdf0ea46a3a641e3bf6400410c0294c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0ea46a3a641e3bf6400410c0294c5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 34921 } } }
I20260807 22:39:28.987565 21920 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdf0ea46a3a641e3bf6400410c0294c5. Latest consensus state: current_term: 1 leader_uuid: "bdf0ea46a3a641e3bf6400410c0294c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf0ea46a3a641e3bf6400410c0294c5" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 34921 } } }
I20260807 22:39:28.987644 21920 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:28.987632 21918 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:28.987891 21929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:28.988417 21929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:28.991535 21929 catalog_manager.cc:1347] Generated new cluster ID: 5ed4289023d34196bc4ccb9ae2769efa
I20260807 22:39:28.991597 21929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:28.997987 21929 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:28.998679 21929 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:29.001055 21929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5: Generated new TSK 0
I20260807 22:39:29.001217 21929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:29.002573 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:34921
--builtin_ntp_servers=127.28.82.212:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:29.100445 21944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:29.100649 21944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:29.100672 21944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:29.101809 21944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:29.101847 21944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:29.101871 21944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:39:29.103276 21944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:29.103497 21944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:29.103812 21944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:39:29.110038 21944 server_base.cc:1034] running on GCE node
W20260807 22:39:29.109990 21954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:29.109987 21952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:29.109993 21951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:29.110497 21944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:29.110765 21944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:29.111963 21944 hybrid_clock.cc:648] HybridClock initialized: now 1786142369111947 us; error 32 us; skew 500 ppm
I20260807 22:39:29.113368 21944 webserver.cc:460] Webserver started at http://127.28.82.193:38257/ using document root <none> and password file <none>
I20260807 22:39:29.113628 21944 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:29.113708 21944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:29.113867 21944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:29.115018 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "597b54b0c00541dd8ee785d5e892d959"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "b93a52c0b8e1fcdd7b5527eb2dc20aca"
server_key_iv: "2e52470c5fc2194a5b0743f3ac24d73b"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.115407 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "597b54b0c00541dd8ee785d5e892d959"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "b93a52c0b8e1fcdd7b5527eb2dc20aca"
server_key_iv: "2e52470c5fc2194a5b0743f3ac24d73b"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.116788 21944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:39:29.117578 21961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.117848 21944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:29.117957 21944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "597b54b0c00541dd8ee785d5e892d959"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "b93a52c0b8e1fcdd7b5527eb2dc20aca"
server_key_iv: "2e52470c5fc2194a5b0743f3ac24d73b"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.118036 21944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:29.133855 21944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:29.134171 21944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:29.134326 21944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:29.134605 21944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:29.135038 21944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:29.135079 21944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.135103 21944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:29.135115 21944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.142741 21944 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:43107
I20260807 22:39:29.142827 22130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:43107 every 8 connection(s)
I20260807 22:39:29.143172 21944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:39:29.148559 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.143912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:49887 (local address 127.28.82.254:34921)
0807 22:39:29.143956 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.143958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.144185 (+ 227us) server_negotiation.cc:407] Connection header received
0807 22:39:29.144770 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.144771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.144785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.144845 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:29.145515 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.146012 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.147106 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.147327 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.147404 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.148276 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:29.148277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:29.148277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:29.148282 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:29.148295 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:29.148299 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:29.148354 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.148456 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.148480 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 22:39:29.148801 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.143898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:34921 (local address 127.28.82.193:49887)
0807 22:39:29.144067 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.144099 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:29.144654 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:29.144910 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:29.144914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:29.144923 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:29.145421 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.145425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.146124 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.146125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.147011 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.147014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.147426 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.147427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.147564 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.148122 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:29.148123 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:29.148150 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:29.148171 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:29.148356 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:29.148362 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:29.148365 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:29.148444 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:39:29.148550 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260807 22:39:29.149341 22131 heartbeater.cc:347] Connected to a master server at 127.28.82.254:34921
I20260807 22:39:29.149447 22131 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:29.149619 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 21944
I20260807 22:39:29.149654 22131 heartbeater.cc:511] Master 127.28.82.254:34921 requested a full tablet report, sending...
I20260807 22:39:29.149734 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:39:29.149953 29003 external_mini_cluster.cc:1428] Setting key 931078ea92cbd6f7517f0dc107e820e0
I20260807 22:39:29.150213 21858 ts_manager.cc:194] Registered new tserver with Master: 597b54b0c00541dd8ee785d5e892d959 (127.28.82.193:43107)
I20260807 22:39:29.150893 21858 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.193:49887
I20260807 22:39:29.151872 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:34921
--builtin_ntp_servers=127.28.82.212:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:29.234511 22136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:29.234702 22136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:29.234724 22136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:29.235885 22136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:29.235925 22136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:29.235950 22136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:39:29.237272 22136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:29.237512 22136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:29.237826 22136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:29.242944 22145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:29.242971 22149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:29.242944 22142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:29.243739 22136 server_base.cc:1034] running on GCE node
I20260807 22:39:29.243995 22136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:29.244257 22136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:29.245433 22136 hybrid_clock.cc:648] HybridClock initialized: now 1786142369245401 us; error 51 us; skew 500 ppm
I20260807 22:39:29.246886 22136 webserver.cc:460] Webserver started at http://127.28.82.194:44311/ using document root <none> and password file <none>
I20260807 22:39:29.247134 22136 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:29.247223 22136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:29.247383 22136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:29.248494 22136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "6f19895e6a2f46969908e028532e26c3"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "f55081248f124876c5e9be1d4033f3b8"
server_key_iv: "f1bc5c89e37f10369c293a7d2d8e6463"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.248867 22136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "6f19895e6a2f46969908e028532e26c3"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "f55081248f124876c5e9be1d4033f3b8"
server_key_iv: "f1bc5c89e37f10369c293a7d2d8e6463"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.250201 22136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:29.251111 22160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.251296 22136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.251398 22136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "6f19895e6a2f46969908e028532e26c3"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "f55081248f124876c5e9be1d4033f3b8"
server_key_iv: "f1bc5c89e37f10369c293a7d2d8e6463"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.251487 22136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:29.282572 22136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:29.282932 22136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:29.283095 22136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:29.283360 22136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:29.283716 22136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:29.283775 22136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.283818 22136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:29.283860 22136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.290354 22136 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:42603
I20260807 22:39:29.290441 22344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:42603 every 8 connection(s)
I20260807 22:39:29.290820 22136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:39:29.295938 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.291640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:54195 (local address 127.28.82.254:34921)
0807 22:39:29.291739 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.291741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.291906 (+ 165us) server_negotiation.cc:407] Connection header received
0807 22:39:29.292406 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.292407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.292418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.292474 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:29.293129 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.293611 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.294664 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.294845 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.294898 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.295738 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:29.295739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:29.295739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:29.295743 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:29.295755 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:29.295758 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:29.295798 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.295859 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.295878 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 22:39:29.296139 22347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.291554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:34921 (local address 127.28.82.194:54195)
0807 22:39:29.291788 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.291822 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:29.292316 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:29.292527 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:29.292530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:29.292538 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:29.293035 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.293039 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.293701 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.293702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.294565 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.294568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.294918 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.294919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.295027 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.295614 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:29.295614 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:29.295642 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:29.295666 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:29.295798 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:29.295800 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:29.295802 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:39:29.295847 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 22:39:29.295922 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260807 22:39:29.296648 22345 heartbeater.cc:347] Connected to a master server at 127.28.82.254:34921
I20260807 22:39:29.296754 22345 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:29.296933 22345 heartbeater.cc:511] Master 127.28.82.254:34921 requested a full tablet report, sending...
I20260807 22:39:29.297350 21857 ts_manager.cc:194] Registered new tserver with Master: 6f19895e6a2f46969908e028532e26c3 (127.28.82.194:42603)
I20260807 22:39:29.297736 21857 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.194:54195
I20260807 22:39:29.297838 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 22136
I20260807 22:39:29.297950 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:39:29.298190 29003 external_mini_cluster.cc:1428] Setting key df7aab0ea538625cefc394376a19d992
I20260807 22:39:29.299103 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:34921
--builtin_ntp_servers=127.28.82.212:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:29.382896 22348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:29.383085 22348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:29.383105 22348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:29.384162 22348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:29.384198 22348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:29.384265 22348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:39:29.385588 22348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:29.385768 22348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:29.386000 22348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:29.391278 22355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:29.391335 22348 server_base.cc:1034] running on GCE node
W20260807 22:39:29.391288 22354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:29.391289 22360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:29.391750 22348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:29.392015 22348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:29.393206 22348 hybrid_clock.cc:648] HybridClock initialized: now 1786142369393203 us; error 37 us; skew 500 ppm
I20260807 22:39:29.394613 22348 webserver.cc:460] Webserver started at http://127.28.82.195:32791/ using document root <none> and password file <none>
I20260807 22:39:29.394860 22348 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:29.394907 22348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:29.395013 22348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:29.396113 22348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "588000e6c78d4031a4110ee18ab41455"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "128c3bb6f205ec39fc94b20e5bc2d3e0"
server_key_iv: "2c147721d2a87060128a192f5b5bb1dc"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.396452 22348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "588000e6c78d4031a4110ee18ab41455"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "128c3bb6f205ec39fc94b20e5bc2d3e0"
server_key_iv: "2c147721d2a87060128a192f5b5bb1dc"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.397876 22348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:29.398717 22374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.398949 22348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:29.399062 22348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "588000e6c78d4031a4110ee18ab41455"
format_stamp: "Formatted at 2026-08-07 22:39:29 on dist-test-slave-7kzw"
server_key: "128c3bb6f205ec39fc94b20e5bc2d3e0"
server_key_iv: "2c147721d2a87060128a192f5b5bb1dc"
server_key_version: "encryptionkey@0"
I20260807 22:39:29.399160 22348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:29.406937 22348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:29.407228 22348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:29.407409 22348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:29.407655 22348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:29.407986 22348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:29.408046 22348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.408094 22348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:29.408141 22348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:29.414708 22348 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:35959
I20260807 22:39:29.414774 22543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:35959 every 8 connection(s)
I20260807 22:39:29.415094 22348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:39:29.420172 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.415845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:34961 (local address 127.28.82.254:34921)
0807 22:39:29.415896 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.415898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.416118 (+ 220us) server_negotiation.cc:407] Connection header received
0807 22:39:29.416567 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.416569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.416584 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.416647 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:29.417357 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.417887 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.418890 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.419079 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.419137 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.419974 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:29.419974 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:29.419975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:29.419978 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:29.419989 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:29.419992 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:29.420030 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.420089 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.420107 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:39:29.420358 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.415814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:34921 (local address 127.28.82.195:34961)
0807 22:39:29.415998 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.416034 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:29.416473 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:29.416696 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:29.416700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:29.416708 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:29.417195 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.417199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.417975 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.417976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.418815 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.418817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.419164 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.419165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.419272 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.419854 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:29.419854 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:29.419882 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:29.419903 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:29.420032 (+ 129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:29.420034 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:29.420036 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:39:29.420080 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 22:39:29.420156 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260807 22:39:29.420886 22544 heartbeater.cc:347] Connected to a master server at 127.28.82.254:34921
I20260807 22:39:29.420990 22544 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:29.421165 22544 heartbeater.cc:511] Master 127.28.82.254:34921 requested a full tablet report, sending...
I20260807 22:39:29.421671 21857 ts_manager.cc:194] Registered new tserver with Master: 588000e6c78d4031a4110ee18ab41455 (127.28.82.195:35959)
I20260807 22:39:29.422072 21857 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.195:34961
I20260807 22:39:29.424942 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 22348
I20260807 22:39:29.425065 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:39:29.425282 29003 external_mini_cluster.cc:1428] Setting key 38a6119cd82fc613d6be982471e8f9ca
I20260807 22:39:29.426445 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:39:29.430815 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.427880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.28.82.254:34921)
0807 22:39:29.427959 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.427962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.428196 (+ 234us) server_negotiation.cc:407] Connection header received
0807 22:39:29.428218 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.428219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.428229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.428269 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:29.428402 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.428847 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.429637 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.429816 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.429880 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.430611 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:29.430612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:29.430613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:29.430617 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:29.430631 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:29.430634 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:29.430675 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.430712 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.430730 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 22:39:29.432421 21857 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:39:29.433931 21857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:39:29.440454 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.436247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:42603 (local address 127.0.0.1:44414)
0807 22:39:29.436402 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.436439 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:29.436499 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:29.437034 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:29.437035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:29.437037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:29.437092 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.437094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.438430 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.438431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.439462 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.439464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.440190 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.440191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.440269 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.440281 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 22:39:29.440351 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:39:29.440400 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260807 22:39:29.440481 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.436352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.28.82.194:42603)
0807 22:39:29.436829 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.436832 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.436841 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:39:29.436875 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.436877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.436909 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.437174 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:29.437239 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.438329 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.439548 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.440099 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.440173 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.440262 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.440361 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.440396 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":112,"threads_started":1}
I20260807 22:39:29.440814 22559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.436254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:35959 (local address 127.0.0.1:56346)
0807 22:39:29.436600 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.436609 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:29.436646 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:29.436991 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:29.436994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:29.436996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:29.437136 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.437138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.438062 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.438063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.439258 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.439260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.440575 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.440576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.440658 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.440662 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:29.440715 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 22:39:29.440749 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260807 22:39:29.441057 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.436576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56346 (local address 127.28.82.195:35959)
0807 22:39:29.436798 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.436802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.436814 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:39:29.436862 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.436864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.436875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.436975 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:29.437225 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.437972 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.439349 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.440105 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.440197 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.440280 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.440922 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.440968 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":98,"threads_started":1}
I20260807 22:39:29.441295 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.436469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43107 (local address 127.0.0.1:35024)
0807 22:39:29.436729 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.436737 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:29.436773 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:29.437309 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:29.437310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:29.437312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:29.437344 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.437345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.438637 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.438638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.439893 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.439896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.441074 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.441076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.441135 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.441140 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:39:29.441190 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 22:39:29.441232 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":77,"threads_started":1}
I20260807 22:39:29.441202 22241 tablet_service.cc:1467] Processing CreateTablet for tablet d7811299b1284ec09adb481300cc03ee (DEFAULT_TABLE table=test-table [id=c705c9350dd94d2fb8190608ad76097e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:29.441584 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.436613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.28.82.193:43107)
0807 22:39:29.437096 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.437100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.437110 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:39:29.437151 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.437153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.437167 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.437240 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:29.437438 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.438558 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.439992 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.440886 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.440952 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.441028 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.441450 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.441496 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":89,"threads_started":1}
I20260807 22:39:29.441601 22241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7811299b1284ec09adb481300cc03ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:29.441649 22449 tablet_service.cc:1467] Processing CreateTablet for tablet d7811299b1284ec09adb481300cc03ee (DEFAULT_TABLE table=test-table [id=c705c9350dd94d2fb8190608ad76097e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:29.441996 22449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7811299b1284ec09adb481300cc03ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:29.442176 22035 tablet_service.cc:1467] Processing CreateTablet for tablet d7811299b1284ec09adb481300cc03ee (DEFAULT_TABLE table=test-table [id=c705c9350dd94d2fb8190608ad76097e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:29.442545 22035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7811299b1284ec09adb481300cc03ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:29.444669 22563 tablet_bootstrap.cc:492] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: Bootstrap starting.
I20260807 22:39:29.445217 22565 tablet_bootstrap.cc:492] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Bootstrap starting.
I20260807 22:39:29.445413 22563 tablet_bootstrap.cc:654] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:29.445751 22563 log.cc:824] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:29.445907 22565 tablet_bootstrap.cc:654] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:29.446238 22565 log.cc:824] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:29.446640 22567 tablet_bootstrap.cc:492] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959: Bootstrap starting.
I20260807 22:39:29.446713 22563 tablet_bootstrap.cc:492] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: No bootstrap required, opened a new log
I20260807 22:39:29.446784 22563 ts_tablet_manager.cc:1397] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:39:29.447026 22565 tablet_bootstrap.cc:492] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: No bootstrap required, opened a new log
I20260807 22:39:29.447103 22565 ts_tablet_manager.cc:1397] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260807 22:39:29.447396 22567 tablet_bootstrap.cc:654] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:29.447866 22567 log.cc:824] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:29.448652 22567 tablet_bootstrap.cc:492] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959: No bootstrap required, opened a new log
I20260807 22:39:29.448729 22567 ts_tablet_manager.cc:1397] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:39:29.449342 22565 raft_consensus.cc:348] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.449492 22565 raft_consensus.cc:374] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:29.449537 22565 raft_consensus.cc:729] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f19895e6a2f46969908e028532e26c3, State: Initialized, Role: FOLLOWER
I20260807 22:39:29.449661 22565 consensus_queue.cc:260] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.449591 22563 raft_consensus.cc:348] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.449741 22563 raft_consensus.cc:374] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:29.449774 22563 raft_consensus.cc:729] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588000e6c78d4031a4110ee18ab41455, State: Initialized, Role: FOLLOWER
I20260807 22:39:29.449903 22563 consensus_queue.cc:260] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.449939 22565 ts_tablet_manager.cc:1428] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 22:39:29.450098 22345 heartbeater.cc:503] Master 127.28.82.254:34921 was elected leader, sending a full tablet report...
I20260807 22:39:29.450161 22563 ts_tablet_manager.cc:1428] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 22:39:29.450196 22544 heartbeater.cc:503] Master 127.28.82.254:34921 was elected leader, sending a full tablet report...
I20260807 22:39:29.451337 22567 raft_consensus.cc:348] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.451501 22567 raft_consensus.cc:374] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:29.451538 22567 raft_consensus.cc:729] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597b54b0c00541dd8ee785d5e892d959, State: Initialized, Role: FOLLOWER
I20260807 22:39:29.451665 22567 consensus_queue.cc:260] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.451900 22567 ts_tablet_manager.cc:1428] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 22:39:29.451956 22131 heartbeater.cc:503] Master 127.28.82.254:34921 was elected leader, sending a full tablet report...
W20260807 22:39:29.541436 22346 tablet.cc:2369] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:39:29.603057 22575 raft_consensus.cc:482] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:29.603200 22575 raft_consensus.cc:504] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.603619 22575 leader_election.cc:288] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f19895e6a2f46969908e028532e26c3 (127.28.82.194:42603), 597b54b0c00541dd8ee785d5e892d959 (127.28.82.193:43107)
I20260807 22:39:29.607468 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.603889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:33783 (local address 127.28.82.194:42603)
0807 22:39:29.603970 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.603972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.603985 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:39:29.604094 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.604095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.604109 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.604180 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:29.604379 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.605125 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.606567 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.607083 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.607159 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.607167 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.607347 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.607391 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 22:39:29.607467 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.603814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:42603 (local address 127.28.82.195:33783)
0807 22:39:29.603892 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.603924 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:29.604004 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:29.604206 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:29.604207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:29.604209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:29.604293 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.604296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.605225 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.605226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.606426 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.606429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.607181 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.607182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.607242 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.607246 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:29.607314 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 22:39:29.607377 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":4}
I20260807 22:39:29.607793 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.604072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:35597 (local address 127.28.82.193:43107)
0807 22:39:29.604171 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.604173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.604373 (+ 200us) server_negotiation.cc:407] Connection header received
0807 22:39:29.604404 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.604406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.604418 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.604478 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:29.604720 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.605375 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.606749 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.607290 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.607383 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.607390 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.607671 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.607727 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 22:39:29.607867 22267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7811299b1284ec09adb481300cc03ee" candidate_uuid: "588000e6c78d4031a4110ee18ab41455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f19895e6a2f46969908e028532e26c3" is_pre_election: true
I20260807 22:39:29.608026 22267 raft_consensus.cc:2454] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 588000e6c78d4031a4110ee18ab41455 in term 0.
I20260807 22:39:29.608058 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.603986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43107 (local address 127.28.82.195:35597)
0807 22:39:29.604266 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.604278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:29.604340 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:29.604518 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:29.604520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:29.604522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:29.604617 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.604618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.605470 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.605471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.606666 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:29.606668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.607417 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.607418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:29.607495 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.607500 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:39:29.607899 (+ 399us) client_negotiation.cc:240] Negotiation successful
0807 22:39:29.607952 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":8,"thread_start_us":103,"threads_started":1}
I20260807 22:39:29.608332 22375 leader_election.cc:302] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588000e6c78d4031a4110ee18ab41455, 6f19895e6a2f46969908e028532e26c3; no voters:
I20260807 22:39:29.608419 22061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7811299b1284ec09adb481300cc03ee" candidate_uuid: "588000e6c78d4031a4110ee18ab41455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "597b54b0c00541dd8ee785d5e892d959" is_pre_election: true
I20260807 22:39:29.608518 22575 raft_consensus.cc:2781] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:39:29.608623 22575 raft_consensus.cc:482] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:39:29.608613 22061 raft_consensus.cc:2454] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 588000e6c78d4031a4110ee18ab41455 in term 0.
I20260807 22:39:29.608650 22575 raft_consensus.cc:3037] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:29.609413 22575 raft_consensus.cc:504] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.609551 22575 leader_election.cc:288] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [CANDIDATE]: Term 1 election: Requested vote from peers 6f19895e6a2f46969908e028532e26c3 (127.28.82.194:42603), 597b54b0c00541dd8ee785d5e892d959 (127.28.82.193:43107)
I20260807 22:39:29.609809 22061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7811299b1284ec09adb481300cc03ee" candidate_uuid: "588000e6c78d4031a4110ee18ab41455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "597b54b0c00541dd8ee785d5e892d959"
I20260807 22:39:29.609791 22267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7811299b1284ec09adb481300cc03ee" candidate_uuid: "588000e6c78d4031a4110ee18ab41455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f19895e6a2f46969908e028532e26c3"
I20260807 22:39:29.609886 22061 raft_consensus.cc:3037] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:29.609886 22267 raft_consensus.cc:3037] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:29.610864 22267 raft_consensus.cc:2454] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 588000e6c78d4031a4110ee18ab41455 in term 1.
I20260807 22:39:29.610864 22061 raft_consensus.cc:2454] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 588000e6c78d4031a4110ee18ab41455 in term 1.
I20260807 22:39:29.611088 22375 leader_election.cc:302] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 588000e6c78d4031a4110ee18ab41455, 597b54b0c00541dd8ee785d5e892d959; no voters:
I20260807 22:39:29.611248 22575 raft_consensus.cc:2781] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:29.611411 22575 raft_consensus.cc:686] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 1 LEADER]: Becoming Leader. State: Replica: 588000e6c78d4031a4110ee18ab41455, State: Running, Role: LEADER
I20260807 22:39:29.611512 22575 consensus_queue.cc:237] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } }
I20260807 22:39:29.612398 21855 catalog_manager.cc:5495] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 588000e6c78d4031a4110ee18ab41455 (127.28.82.195). New cstate: current_term: 1 leader_uuid: "588000e6c78d4031a4110ee18ab41455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "588000e6c78d4031a4110ee18ab41455" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 35959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 } health_report { overall_health: UNKNOWN } } }
I20260807 22:39:29.624732 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.622303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.28.82.195:35959)
0807 22:39:29.622362 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:29.622364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:29.622388 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:39:29.622486 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:29.622487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:29.622510 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:29.622584 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:39:29.622727 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.623192 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.623947 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:29.624141 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:29.624223 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:29.624394 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:39:29.624485 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:39:29.624571 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:29.624622 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 22:39:29.624639 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:39:29.626571 22061 raft_consensus.cc:1261] T d7811299b1284ec09adb481300cc03ee P 597b54b0c00541dd8ee785d5e892d959 [term 1 FOLLOWER]: Refusing update from remote peer 588000e6c78d4031a4110ee18ab41455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:29.626570 22267 raft_consensus.cc:1261] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 1 FOLLOWER]: Refusing update from remote peer 588000e6c78d4031a4110ee18ab41455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:29.626958 22575 consensus_queue.cc:1035] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f19895e6a2f46969908e028532e26c3" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:29.627104 22578 consensus_queue.cc:1035] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "597b54b0c00541dd8ee785d5e892d959" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:29.629619 22582 mvcc.cc:205] Tried to move back new op lower bound from 7316039145987710976 to 7316039145929592832. Current Snapshot: MvccSnapshot[applied={T|T < 7316039145987710976}]
I20260807 22:39:29.630275 22585 mvcc.cc:205] Tried to move back new op lower bound from 7316039145987710976 to 7316039145929592832. Current Snapshot: MvccSnapshot[applied={T|T < 7316039145987710976}]
I20260807 22:39:29.630868 22587 mvcc.cc:205] Tried to move back new op lower bound from 7316039145987710976 to 7316039145929592832. Current Snapshot: MvccSnapshot[applied={T|T < 7316039145987710976 or (T in {7316039145987710976})}]
I20260807 22:39:29.631990 21855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:29.632094 21855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:29.632869 21855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdf0ea46a3a641e3bf6400410c0294c5: Sending DeleteTablet for 3 replicas of tablet d7811299b1284ec09adb481300cc03ee
I20260807 22:39:29.633193 22241 tablet_service.cc:1514] Processing DeleteTablet for tablet d7811299b1284ec09adb481300cc03ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:29 UTC) from {username='slave'} at 127.0.0.1:44414
I20260807 22:39:29.633226 22035 tablet_service.cc:1514] Processing DeleteTablet for tablet d7811299b1284ec09adb481300cc03ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:29 UTC) from {username='slave'} at 127.0.0.1:35024
I20260807 22:39:29.633448 22449 tablet_service.cc:1514] Processing DeleteTablet for tablet d7811299b1284ec09adb481300cc03ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:29 UTC) from {username='slave'} at 127.0.0.1:56346
I20260807 22:39:29.633508 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 21944
I20260807 22:39:29.657495 22600 tablet_replica.cc:330] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: stopping tablet replica
I20260807 22:39:29.657698 22600 raft_consensus.cc:2229] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:29.657841 22600 raft_consensus.cc:2258] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:29.658298 22605 tablet_replica.cc:330] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: stopping tablet replica
I20260807 22:39:29.658398 22605 raft_consensus.cc:2229] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:39:29.658514 22605 raft_consensus.cc:2258] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:29.659966 22600 ts_tablet_manager.cc:1905] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:29.662060 22600 ts_tablet_manager.cc:1918] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:29.662138 22600 log.cc:1196] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/d7811299b1284ec09adb481300cc03ee
I20260807 22:39:29.662388 22600 ts_tablet_manager.cc:1939] T d7811299b1284ec09adb481300cc03ee P 6f19895e6a2f46969908e028532e26c3: Deleting consensus metadata
I20260807 22:39:29.663757 22605 ts_tablet_manager.cc:1905] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:29.666028 22605 ts_tablet_manager.cc:1918] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:29.666189 22605 log.cc:1196] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/d7811299b1284ec09adb481300cc03ee
I20260807 22:39:29.666563 22605 ts_tablet_manager.cc:1939] T d7811299b1284ec09adb481300cc03ee P 588000e6c78d4031a4110ee18ab41455: Deleting consensus metadata
W20260807 22:39:29.666754 21831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:43107 (error 108)
I20260807 22:39:29.667125 21836 catalog_manager.cc:4841] TS 588000e6c78d4031a4110ee18ab41455 (127.28.82.195:35959): tablet d7811299b1284ec09adb481300cc03ee (table test-table [id=c705c9350dd94d2fb8190608ad76097e]) successfully deleted
I20260807 22:39:29.667256 21831 catalog_manager.cc:4841] TS 6f19895e6a2f46969908e028532e26c3 (127.28.82.194:42603): tablet d7811299b1284ec09adb481300cc03ee (table test-table [id=c705c9350dd94d2fb8190608ad76097e]) successfully deleted
I20260807 22:39:29.667856 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 22136
I20260807 22:39:29.667902 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:29.667711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43107 (local address 127.0.0.1:35038)
0807 22:39:29.667785 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:29.667864 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 22:39:29.673486 21831 catalog_manager.cc:4584] TS 597b54b0c00541dd8ee785d5e892d959 (127.28.82.193:43107): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7811299b1284ec09adb481300cc03ee: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43107: connect: Connection refused (error 111)
I20260807 22:39:29.678970 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 22348
I20260807 22:39:29.691062 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 21811
2026-08-07T22:39:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (859 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:29 dist-test-slave-7kzw krb5kdc[22612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:29 dist-test-slave-7kzw krb5kdc[22612](info): set up 2 sockets
Aug 07 22:39:29 dist-test-slave-7kzw krb5kdc[22612](info): commencing operation
krb5kdc: starting...
W20260807 22:39:30.938738 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:39:31.063408 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39157
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:31.146999 22629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:31.147238 22629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:31.147292 22629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:31.148427 22629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:31.148495 22629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:31.148521 22629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:31.148561 22629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:31.148590 22629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:31.150063 22629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:31.150336 22629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:31.150615 22629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:31.155505 22636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:31.155506 22635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:31.155494 22639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:31.155570 22629 server_base.cc:1034] running on GCE node
I20260807 22:39:31.155979 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:31.156267 22629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:31.157449 22629 hybrid_clock.cc:648] HybridClock initialized: now 1786142371157433 us; error 26 us; skew 500 ppm
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:31.160137 22629 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:39:31.160497 22629 webserver.cc:460] Webserver started at http://127.28.82.254:41809/ using document root <none> and password file <none>
I20260807 22:39:31.160691 22629 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:31.160732 22629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:31.160820 22629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:31.161754 22629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "99ee0736b9294dcaa29ae8b2dc91116c"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "c3ff3cf12536506861f1e5f1233b12ed"
server_key_iv: "9b18cad237df6ecf307fef00dc53afce"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.162050 22629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "99ee0736b9294dcaa29ae8b2dc91116c"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "c3ff3cf12536506861f1e5f1233b12ed"
server_key_iv: "9b18cad237df6ecf307fef00dc53afce"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.163386 22629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 22:39:31.164136 22650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.164322 22629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:31.164414 22629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "99ee0736b9294dcaa29ae8b2dc91116c"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "c3ff3cf12536506861f1e5f1233b12ed"
server_key_iv: "9b18cad237df6ecf307fef00dc53afce"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.164487 22629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:31.195468 22629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:31.197837 22629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:31.197981 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:31.202229 22629 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:39157
I20260807 22:39:31.202293 22728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:39157 every 8 connection(s)
I20260807 22:39:31.202674 22629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:31.203722 22729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:31.206041 22729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c: Bootstrap starting.
I20260807 22:39:31.206696 22729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:31.207010 22729 log.cc:824] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:31.207823 22729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c: No bootstrap required, opened a new log
I20260807 22:39:31.209113 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 22629
I20260807 22:39:31.209271 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:31.209468 29003 external_mini_cluster.cc:1428] Setting key e9d516db0f1c7a424bdbcfdb091138c7
I20260807 22:39:31.209794 22729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee0736b9294dcaa29ae8b2dc91116c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39157 } }
I20260807 22:39:31.209980 22729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:31.210026 22729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ee0736b9294dcaa29ae8b2dc91116c, State: Initialized, Role: FOLLOWER
I20260807 22:39:31.210192 22729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee0736b9294dcaa29ae8b2dc91116c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39157 } }
I20260807 22:39:31.210314 22729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:31.210362 22729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:31.210421 22729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:31.211449 22729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee0736b9294dcaa29ae8b2dc91116c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39157 } }
I20260807 22:39:31.211594 22729 leader_election.cc:302] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99ee0736b9294dcaa29ae8b2dc91116c; no voters:
I20260807 22:39:31.211820 22729 leader_election.cc:288] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:31.211862 22736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:31.212008 22736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [term 1 LEADER]: Becoming Leader. State: Replica: 99ee0736b9294dcaa29ae8b2dc91116c, State: Running, Role: LEADER
I20260807 22:39:31.212117 22736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee0736b9294dcaa29ae8b2dc91116c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39157 } }
I20260807 22:39:31.212105 22729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:31.212587 22741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99ee0736b9294dcaa29ae8b2dc91116c. Latest consensus state: current_term: 1 leader_uuid: "99ee0736b9294dcaa29ae8b2dc91116c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee0736b9294dcaa29ae8b2dc91116c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39157 } } }
I20260807 22:39:31.212677 22741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:31.212841 22740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99ee0736b9294dcaa29ae8b2dc91116c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee0736b9294dcaa29ae8b2dc91116c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39157 } } }
I20260807 22:39:31.212893 22740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:31.213223 22745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:31.213709 22745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:31.216792 22745 catalog_manager.cc:1347] Generated new cluster ID: fe58682e25f44a879944501ab63b5fa7
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:31.216849 22745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:31.220726 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.210033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44418 (local address 127.28.82.254:39157)
0807 22:39:31.210220 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.210236 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.210262 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:39:31.210376 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.210383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.210438 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.210591 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:31.211232 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.212403 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.213464 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.214328 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.214507 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.217336 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:31.217340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:31.217340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:31.217352 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:31.219146 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.219537 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.219540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.219540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.219658 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.219880 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.219881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.219881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.220112 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:31.220274 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.220383 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.220522 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260807 22:39:31.223162 22745 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:31.223677 22745 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:31.230188 22745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c: Generated new TSK 0
I20260807 22:39:31.230369 22745 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:39:31.261200 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39157
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:31.343997 22769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:31.344203 22769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:31.344234 22769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:31.345302 22769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:31.345337 22769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:31.345395 22769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:39:31.346864 22769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:31.347046 22769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:31.347220 22769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:31.352360 22774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:31.352377 22769 server_base.cc:1034] running on GCE node
W20260807 22:39:31.352360 22777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:31.352387 22775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:31.352804 22769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:31.353065 22769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:31.354233 22769 hybrid_clock.cc:648] HybridClock initialized: now 1786142371354219 us; error 29 us; skew 500 ppm
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:31.356886 22769 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:39:31.357368 22769 webserver.cc:460] Webserver started at http://127.28.82.193:40503/ using document root <none> and password file <none>
I20260807 22:39:31.357589 22769 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:31.357656 22769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:31.357807 22769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:31.358902 22769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "b1d595901e7c4e07a6acc7991b177469"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "ff22c46ce62422a2bb619c96b974de97"
server_key_iv: "b3c7987546b15eb7bc47ff468e3c11cd"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.359285 22769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "b1d595901e7c4e07a6acc7991b177469"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "ff22c46ce62422a2bb619c96b974de97"
server_key_iv: "b3c7987546b15eb7bc47ff468e3c11cd"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.360581 22769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:31.361320 22787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.361490 22769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:31.361599 22769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "b1d595901e7c4e07a6acc7991b177469"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "ff22c46ce62422a2bb619c96b974de97"
server_key_iv: "b3c7987546b15eb7bc47ff468e3c11cd"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.361672 22769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:31.370937 22769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:31.372745 22769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:31.372902 22769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:31.373128 22769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:31.373442 22769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:31.373497 22769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.373539 22769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:31.373567 22769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.379850 22769 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:37801
I20260807 22:39:31.379907 22970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:37801 every 8 connection(s)
I20260807 22:39:31.380211 22769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:39:31.386066 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 22769
I20260807 22:39:31.386200 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:31.386430 29003 external_mini_cluster.cc:1428] Setting key d508ee46cc0e0888914bb6bc935ef4bd
I20260807 22:39:31.390142 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.380935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:40593 (local address 127.28.82.254:39157)
0807 22:39:31.381007 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.381009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.381226 (+ 217us) server_negotiation.cc:407] Connection header received
0807 22:39:31.381767 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.381769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.381786 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.381844 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:31.382821 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.383343 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.384333 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.384558 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.384636 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.387022 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:31.387023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:31.387023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:31.387029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:31.388668 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.389131 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.389133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.389134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.389208 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.389471 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.389471 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.389472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.389591 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:31.389661 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.389965 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.390054 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:39:31.390370 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.380922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39157 (local address 127.28.82.193:40593)
0807 22:39:31.381101 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.381137 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:31.381654 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:31.381913 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:31.381917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:31.382520 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:31.382716 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.382721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.383443 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.383444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.384237 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.384239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.384657 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.384658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.384791 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.385245 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:31.385254 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:31.386944 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:31.388782 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:31.388789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:31.388791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:31.389021 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:31.389279 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:31.389280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:31.389280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:31.389390 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:31.389698 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:31.389705 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:31.389829 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 22:39:31.389938 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 22:39:31.390090 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260807 22:39:31.390861 22971 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39157
I20260807 22:39:31.390951 22971 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:31.391144 22971 heartbeater.cc:511] Master 127.28.82.254:39157 requested a full tablet report, sending...
I20260807 22:39:31.391641 22675 ts_manager.cc:194] Registered new tserver with Master: b1d595901e7c4e07a6acc7991b177469 (127.28.82.193:37801)
I20260807 22:39:31.392249 22675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:40593
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:39:31.413313 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39157
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:31.495852 22984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:31.496054 22984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:31.496078 22984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:31.497211 22984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:31.497257 22984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:31.497283 22984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:39:31.498754 22984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:31.499012 22984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:31.499248 22984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:31.504493 22993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:31.504561 22992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:31.504591 22984 server_base.cc:1034] running on GCE node
W20260807 22:39:31.504575 22995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:31.504850 22984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:31.505045 22984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:31.506183 22984 hybrid_clock.cc:648] HybridClock initialized: now 1786142371506165 us; error 35 us; skew 500 ppm
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:31.508735 22984 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:39:31.509199 22984 webserver.cc:460] Webserver started at http://127.28.82.194:45647/ using document root <none> and password file <none>
I20260807 22:39:31.509426 22984 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:31.509472 22984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:31.509639 22984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:31.510730 22984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "e64fc0fec6ac4fedadc7a7945288538c"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "9b2946394b746a530d8afa05eac6a8a1"
server_key_iv: "a7bb6ae754e0e445f10c19af66511e47"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.511090 22984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "e64fc0fec6ac4fedadc7a7945288538c"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "9b2946394b746a530d8afa05eac6a8a1"
server_key_iv: "a7bb6ae754e0e445f10c19af66511e47"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.512467 22984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:39:31.513286 23011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.513458 22984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:31.513562 22984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "e64fc0fec6ac4fedadc7a7945288538c"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "9b2946394b746a530d8afa05eac6a8a1"
server_key_iv: "a7bb6ae754e0e445f10c19af66511e47"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.513645 22984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:31.525857 22984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:31.527804 22984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:31.527964 22984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:31.528193 22984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:31.528505 22984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:31.528560 22984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.528602 22984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:31.528630 22984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.534751 22984 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:46645
I20260807 22:39:31.534818 23185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:46645 every 8 connection(s)
I20260807 22:39:31.535157 22984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:39:31.538312 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 22984
I20260807 22:39:31.538463 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:39:31.538676 29003 external_mini_cluster.cc:1428] Setting key b1036c13615e407927a0d02fc0ec828b
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:31.545215 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.535924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39157 (local address 127.28.82.194:53359)
0807 22:39:31.536120 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.536154 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:31.536665 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:31.536908 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:31.536912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:31.537503 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:31.537703 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.537707 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.538433 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.538434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.539624 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.539627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.540014 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.540015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.540154 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.540663 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:31.540671 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:31.542060 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:31.543893 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:31.543900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:31.543902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:31.544121 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:31.544345 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:31.544345 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:31.544346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:31.544445 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:31.544732 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:31.544737 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:31.544860 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 22:39:31.544946 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 22:39:31.545061 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
I20260807 22:39:31.545233 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.535950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:53359 (local address 127.28.82.254:39157)
0807 22:39:31.536040 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.536043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.536238 (+ 195us) server_negotiation.cc:407] Connection header received
0807 22:39:31.536774 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.536776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.536790 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.536842 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:31.537812 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.538334 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.539726 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.539920 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.540005 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.542154 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:31.542156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:31.542156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:31.542164 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:31.543794 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.544200 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.544201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.544202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.544274 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.544510 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.544511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.544511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.544663 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:31.544736 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.544996 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.545098 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 22:39:31.545961 23187 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39157
I20260807 22:39:31.546047 23187 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:31.546242 23187 heartbeater.cc:511] Master 127.28.82.254:39157 requested a full tablet report, sending...
I20260807 22:39:31.546696 22675 ts_manager.cc:194] Registered new tserver with Master: e64fc0fec6ac4fedadc7a7945288538c (127.28.82.194:46645)
I20260807 22:39:31.547217 22675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:53359
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:39:31.571341 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39157
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:31.654938 23193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:31.655141 23193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:31.655164 23193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:31.656453 23193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:31.656505 23193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:31.656535 23193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:39:31.658054 23193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:31.658246 23193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:31.658427 23193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:31.664143 23205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:31.664233 23208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:31.664156 23203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:31.664804 23193 server_base.cc:1034] running on GCE node
I20260807 22:39:31.665036 23193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:31.665279 23193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:31.666429 23193 hybrid_clock.cc:648] HybridClock initialized: now 1786142371666408 us; error 37 us; skew 500 ppm
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:31.669549 23193 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:39:31.669975 23193 webserver.cc:460] Webserver started at http://127.28.82.195:36425/ using document root <none> and password file <none>
I20260807 22:39:31.670224 23193 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:31.670271 23193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:31.670416 23193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:31.671466 23193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "06012df9a46f450ab44dfde6e04f915b"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "5307c8c94a28eca5f69910739d10adb4"
server_key_iv: "b9fb0c29bbf0c4981c4e83ab19a5f049"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.671823 23193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "06012df9a46f450ab44dfde6e04f915b"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "5307c8c94a28eca5f69910739d10adb4"
server_key_iv: "b9fb0c29bbf0c4981c4e83ab19a5f049"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.673249 23193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:39:31.674110 23221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.674304 23193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:31.674446 23193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "06012df9a46f450ab44dfde6e04f915b"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "5307c8c94a28eca5f69910739d10adb4"
server_key_iv: "b9fb0c29bbf0c4981c4e83ab19a5f049"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.674563 23193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:31.692340 23193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:31.694628 23193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:31.694803 23193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:31.695043 23193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:31.695390 23193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:31.695449 23193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.695494 23193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:31.695521 23193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.701478 23193 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:38461
I20260807 22:39:31.701553 23398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:38461 every 8 connection(s)
I20260807 22:39:31.701874 23193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:39:31.706955 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 23193
I20260807 22:39:31.707062 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:39:31.707284 29003 external_mini_cluster.cc:1428] Setting key 792de2e36002c68fdcb33a59b73a879e
Aug 07 22:39:31 dist-test-slave-7kzw krb5kdc[22612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142371, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:31.711831 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.702715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:43787 (local address 127.28.82.254:39157)
0807 22:39:31.702810 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.702812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.702992 (+ 180us) server_negotiation.cc:407] Connection header received
0807 22:39:31.703451 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.703452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.703462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.703509 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:31.704433 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.704924 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.705901 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.706092 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.706158 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.708787 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:31.708789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:31.708789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:31.708798 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:31.710282 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.710805 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.710807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.710807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.710880 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.711153 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.711154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.711155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.711277 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:31.711357 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.711643 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.711742 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:39:31.712004 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.702686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39157 (local address 127.28.82.195:43787)
0807 22:39:31.702862 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.702897 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:31.703366 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:31.703568 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:31.703571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:31.704157 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:31.704339 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.704344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.705009 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.705010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.705801 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.705803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.706181 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.706182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.706292 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.706851 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:31.706860 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:31.708680 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:31.710389 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:31.710398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:31.710400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:31.710723 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:31.710954 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:31.710955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:31.710955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:31.711076 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:31.711348 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:31.711355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:31.711499 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 22:39:31.711605 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 22:39:31.711767 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260807 22:39:31.712544 23399 heartbeater.cc:347] Connected to a master server at 127.28.82.254:39157
I20260807 22:39:31.712652 23399 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:31.712816 23399 heartbeater.cc:511] Master 127.28.82.254:39157 requested a full tablet report, sending...
I20260807 22:39:31.713248 22675 ts_manager.cc:194] Registered new tserver with Master: 06012df9a46f450ab44dfde6e04f915b (127.28.82.195:38461)
I20260807 22:39:31.713650 22675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:43787
I20260807 22:39:31.719842 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:39:31.727118 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.721247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.28.82.254:39157)
0807 22:39:31.721306 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.721308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.721419 (+ 111us) server_negotiation.cc:407] Connection header received
0807 22:39:31.721457 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.721458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.721466 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.721520 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:31.722032 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.722533 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.723221 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.723392 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.723441 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.724632 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:31.724634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:31.724634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:31.724640 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:31.726020 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.726378 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.726379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.726379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.726431 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:31.726678 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:31.726679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:31.726679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:31.726823 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:31.726903 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.727013 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.727060 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 22:39:31.728713 22675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:39:31.730387 22675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:39:31.736919 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.732696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:37801 (local address 127.0.0.1:57654)
0807 22:39:31.733037 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.733082 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:31.733132 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:31.733439 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:31.733441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:31.733444 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:31.733535 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.733536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.734566 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.734567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.735670 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.735672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.736593 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.736594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.736684 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.736698 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:39:31.736774 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 22:39:31.736824 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":2,"thread_start_us":58,"threads_started":1}
I20260807 22:39:31.736925 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.732816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.28.82.193:37801)
0807 22:39:31.733296 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.733300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.733308 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:39:31.733341 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.733343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.733355 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.733419 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:31.733631 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.734428 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.735767 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.736506 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.736581 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.736638 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.736799 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.736841 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":97,"threads_started":1}
I20260807 22:39:31.737255 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.732696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38461 (local address 127.0.0.1:34176)
0807 22:39:31.733034 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.733077 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:31.733140 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:31.733700 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:31.733701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:31.733704 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:31.733751 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.733753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.735054 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.735055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.736108 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.736111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.737000 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.737001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.737068 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.737071 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:31.737154 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 22:39:31.737195 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":74,"threads_started":1}
I20260807 22:39:31.737301 23417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.732866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.28.82.195:38461)
0807 22:39:31.733559 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.733563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.733572 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:39:31.733610 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.733612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.733628 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.733701 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:31.733837 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.734937 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.736196 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.736848 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.736910 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.736962 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.737177 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.737217 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":96,"threads_started":1}
I20260807 22:39:31.737623 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.732893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:46645 (local address 127.0.0.1:60530)
0807 22:39:31.733785 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.733795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:31.733843 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:31.734128 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:31.734129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:31.734131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:31.734165 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.734167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.735324 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.735325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.736262 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.736265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.737303 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.737304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.737451 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.737455 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:31.737510 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 22:39:31.737546 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"thread_start_us":59,"threads_started":1}
I20260807 22:39:31.737696 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.733021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.28.82.194:46645)
0807 22:39:31.733247 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.733251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.733905 (+ 654us) server_negotiation.cc:407] Connection header received
0807 22:39:31.733949 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.733951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.733966 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.734046 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:31.734239 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.735229 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.736353 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.737148 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.737215 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.737270 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.737556 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.737603 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":122,"threads_started":1}
I20260807 22:39:31.738021 23298 tablet_service.cc:1467] Processing CreateTablet for tablet 55b4f66b9fdf47a79989495eac56488c (DEFAULT_TABLE table=test-table [id=29e8e1961a854553b3d4864ea1f920cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:31.738276 23084 tablet_service.cc:1467] Processing CreateTablet for tablet 55b4f66b9fdf47a79989495eac56488c (DEFAULT_TABLE table=test-table [id=29e8e1961a854553b3d4864ea1f920cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:31.738430 23298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b4f66b9fdf47a79989495eac56488c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:31.738658 23084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b4f66b9fdf47a79989495eac56488c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:31.741146 23419 tablet_bootstrap.cc:492] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: Bootstrap starting.
I20260807 22:39:31.741268 23422 tablet_bootstrap.cc:492] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Bootstrap starting.
I20260807 22:39:31.741837 23422 tablet_bootstrap.cc:654] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:31.741930 23419 tablet_bootstrap.cc:654] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:31.741882 22867 tablet_service.cc:1467] Processing CreateTablet for tablet 55b4f66b9fdf47a79989495eac56488c (DEFAULT_TABLE table=test-table [id=29e8e1961a854553b3d4864ea1f920cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:31.742254 22867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b4f66b9fdf47a79989495eac56488c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:31.742453 23422 log.cc:824] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:31.742552 23419 log.cc:824] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:31.743331 23422 tablet_bootstrap.cc:492] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: No bootstrap required, opened a new log
I20260807 22:39:31.743413 23422 ts_tablet_manager.cc:1397] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:39:31.745486 23422 raft_consensus.cc:348] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.745677 23422 raft_consensus.cc:374] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:31.745733 23422 raft_consensus.cc:729] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64fc0fec6ac4fedadc7a7945288538c, State: Initialized, Role: FOLLOWER
I20260807 22:39:31.745828 23426 tablet_bootstrap.cc:492] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469: Bootstrap starting.
I20260807 22:39:31.745842 23422 consensus_queue.cc:260] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.746153 23422 ts_tablet_manager.cc:1428] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 22:39:31.746186 23187 heartbeater.cc:503] Master 127.28.82.254:39157 was elected leader, sending a full tablet report...
I20260807 22:39:31.746569 23426 tablet_bootstrap.cc:654] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:31.746616 23419 tablet_bootstrap.cc:492] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: No bootstrap required, opened a new log
I20260807 22:39:31.746685 23419 ts_tablet_manager.cc:1397] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260807 22:39:31.747257 23426 log.cc:824] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:31.747956 23426 tablet_bootstrap.cc:492] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469: No bootstrap required, opened a new log
I20260807 22:39:31.748039 23426 ts_tablet_manager.cc:1397] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:39:31.748394 23419 raft_consensus.cc:348] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.748575 23419 raft_consensus.cc:374] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:31.748626 23419 raft_consensus.cc:729] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06012df9a46f450ab44dfde6e04f915b, State: Initialized, Role: FOLLOWER
I20260807 22:39:31.748869 23419 consensus_queue.cc:260] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.749167 23419 ts_tablet_manager.cc:1428] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 22:39:31.749214 23399 heartbeater.cc:503] Master 127.28.82.254:39157 was elected leader, sending a full tablet report...
I20260807 22:39:31.749986 23426 raft_consensus.cc:348] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.750182 23426 raft_consensus.cc:374] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:31.750221 23426 raft_consensus.cc:729] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d595901e7c4e07a6acc7991b177469, State: Initialized, Role: FOLLOWER
I20260807 22:39:31.750329 23426 consensus_queue.cc:260] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.750618 23426 ts_tablet_manager.cc:1428] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 22:39:31.750699 22971 heartbeater.cc:503] Master 127.28.82.254:39157 was elected leader, sending a full tablet report...
I20260807 22:39:31.780962 23433 raft_consensus.cc:482] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:31.781126 23433 raft_consensus.cc:504] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.781502 23433 leader_election.cc:288] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06012df9a46f450ab44dfde6e04f915b (127.28.82.195:38461), e64fc0fec6ac4fedadc7a7945288538c (127.28.82.194:46645)
I20260807 22:39:31.784668 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.781926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:53075 (local address 127.28.82.194:46645)
0807 22:39:31.781997 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.781999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.782101 (+ 102us) server_negotiation.cc:407] Connection header received
0807 22:39:31.782149 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.782150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.782160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.782220 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:31.782399 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.782898 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.783912 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.784295 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.784351 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.784358 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.784539 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.784592 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:39:31.784709 23417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.781838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:49033 (local address 127.28.82.195:38461)
0807 22:39:31.781965 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.781968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.781977 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:39:31.782007 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.782007 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.782019 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.782078 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:31.782284 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.782915 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.783913 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.784312 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.784401 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.784410 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.784596 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.784639 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 22:39:31.784687 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.781687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:38461 (local address 127.28.82.193:49033)
0807 22:39:31.781754 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.781768 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:31.781844 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:31.782124 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:31.782125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:31.782127 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:31.782205 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.782207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.783034 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.783035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.783810 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.783812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.784431 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.784432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.784512 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.784516 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:31.784576 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 22:39:31.784611 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":37}
I20260807 22:39:31.784680 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.781813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:46645 (local address 127.28.82.193:53075)
0807 22:39:31.782008 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.782019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:31.782086 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:31.782231 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:31.782232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:31.782233 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:31.782325 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.782327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.782984 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.782985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.783813 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:31.783815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.784382 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.784384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:31.784463 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.784467 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:31.784509 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 22:39:31.784543 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":1,"thread_start_us":121,"threads_started":1}
I20260807 22:39:31.785092 23322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b4f66b9fdf47a79989495eac56488c" candidate_uuid: "b1d595901e7c4e07a6acc7991b177469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06012df9a46f450ab44dfde6e04f915b" is_pre_election: true
I20260807 22:39:31.785250 23322 raft_consensus.cc:2454] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d595901e7c4e07a6acc7991b177469 in term 0.
I20260807 22:39:31.785517 22789 leader_election.cc:302] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06012df9a46f450ab44dfde6e04f915b, b1d595901e7c4e07a6acc7991b177469; no voters:
I20260807 22:39:31.785702 23119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b4f66b9fdf47a79989495eac56488c" candidate_uuid: "b1d595901e7c4e07a6acc7991b177469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64fc0fec6ac4fedadc7a7945288538c" is_pre_election: true
I20260807 22:39:31.785753 23433 raft_consensus.cc:2781] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260807 22:39:31.785763 23188 tablet.cc:2369] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:39:31.785812 23433 raft_consensus.cc:482] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:39:31.785840 23433 raft_consensus.cc:3037] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:31.785868 23119 raft_consensus.cc:2454] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d595901e7c4e07a6acc7991b177469 in term 0.
I20260807 22:39:31.786751 23433 raft_consensus.cc:504] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.786860 23433 leader_election.cc:288] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [CANDIDATE]: Term 1 election: Requested vote from peers 06012df9a46f450ab44dfde6e04f915b (127.28.82.195:38461), e64fc0fec6ac4fedadc7a7945288538c (127.28.82.194:46645)
I20260807 22:39:31.787035 23322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b4f66b9fdf47a79989495eac56488c" candidate_uuid: "b1d595901e7c4e07a6acc7991b177469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06012df9a46f450ab44dfde6e04f915b"
I20260807 22:39:31.787088 23119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b4f66b9fdf47a79989495eac56488c" candidate_uuid: "b1d595901e7c4e07a6acc7991b177469" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e64fc0fec6ac4fedadc7a7945288538c"
I20260807 22:39:31.787135 23322 raft_consensus.cc:3037] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:31.787200 23119 raft_consensus.cc:3037] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:31.787910 23322 raft_consensus.cc:2454] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d595901e7c4e07a6acc7991b177469 in term 1.
I20260807 22:39:31.788060 23119 raft_consensus.cc:2454] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d595901e7c4e07a6acc7991b177469 in term 1.
I20260807 22:39:31.788100 22789 leader_election.cc:302] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06012df9a46f450ab44dfde6e04f915b, b1d595901e7c4e07a6acc7991b177469; no voters:
I20260807 22:39:31.788276 23433 raft_consensus.cc:2781] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:31.788420 23433 raft_consensus.cc:686] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 1 LEADER]: Becoming Leader. State: Replica: b1d595901e7c4e07a6acc7991b177469, State: Running, Role: LEADER
I20260807 22:39:31.788523 23433 consensus_queue.cc:237] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } }
I20260807 22:39:31.789379 22675 catalog_manager.cc:5495] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1d595901e7c4e07a6acc7991b177469 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "b1d595901e7c4e07a6acc7991b177469" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d595901e7c4e07a6acc7991b177469" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 37801 } health_report { overall_health: HEALTHY } } }
I20260807 22:39:31.795145 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.792574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.28.82.193:37801)
0807 22:39:31.792643 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.792645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.792657 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:39:31.792692 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.792693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.792712 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.792789 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:39:31.792900 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.793425 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.794230 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.794408 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.794525 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.794696 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:39:31.794778 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:39:31.794857 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.795021 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.795035 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:39:31.797096 23119 raft_consensus.cc:1261] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 1 FOLLOWER]: Refusing update from remote peer b1d595901e7c4e07a6acc7991b177469: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:31.797096 23322 raft_consensus.cc:1261] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 1 FOLLOWER]: Refusing update from remote peer b1d595901e7c4e07a6acc7991b177469: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:31.797462 23433 consensus_queue.cc:1035] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06012df9a46f450ab44dfde6e04f915b" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:31.797613 23435 consensus_queue.cc:1035] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e64fc0fec6ac4fedadc7a7945288538c" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 46645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:31.800621 23444 mvcc.cc:205] Tried to move back new op lower bound from 7316039154877468672 to 7316039154846576640. Current Snapshot: MvccSnapshot[applied={T|T < 7316039154877468672}]
I20260807 22:39:31.802076 22675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:31.802181 22675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:31.802985 22675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99ee0736b9294dcaa29ae8b2dc91116c: Sending DeleteTablet for 3 replicas of tablet 55b4f66b9fdf47a79989495eac56488c
I20260807 22:39:31.803332 22867 tablet_service.cc:1514] Processing DeleteTablet for tablet 55b4f66b9fdf47a79989495eac56488c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:31 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:57654
I20260807 22:39:31.803431 23298 tablet_service.cc:1514] Processing DeleteTablet for tablet 55b4f66b9fdf47a79989495eac56488c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:31 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:34176
I20260807 22:39:31.803591 23456 tablet_replica.cc:330] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469: stopping tablet replica
I20260807 22:39:31.803606 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 22769
I20260807 22:39:31.803679 23084 tablet_service.cc:1514] Processing DeleteTablet for tablet 55b4f66b9fdf47a79989495eac56488c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:31 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:60530
I20260807 22:39:31.803728 23456 raft_consensus.cc:2229] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:39:31.803877 23456 raft_consensus.cc:2258] T 55b4f66b9fdf47a79989495eac56488c P b1d595901e7c4e07a6acc7991b177469 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:31.804379 23458 tablet_replica.cc:330] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: stopping tablet replica
I20260807 22:39:31.804484 23458 raft_consensus.cc:2229] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:31.804576 23458 raft_consensus.cc:2258] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:31.804711 23457 tablet_replica.cc:330] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: stopping tablet replica
I20260807 22:39:31.804802 23457 raft_consensus.cc:2229] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:31.804885 23457 raft_consensus.cc:2258] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:31.806521 23458 ts_tablet_manager.cc:1905] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:31.808704 23458 ts_tablet_manager.cc:1918] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:31.808777 23458 log.cc:1196] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/55b4f66b9fdf47a79989495eac56488c
I20260807 22:39:31.809062 23458 ts_tablet_manager.cc:1939] T 55b4f66b9fdf47a79989495eac56488c P e64fc0fec6ac4fedadc7a7945288538c: Deleting consensus metadata
I20260807 22:39:31.809525 22652 catalog_manager.cc:4841] TS e64fc0fec6ac4fedadc7a7945288538c (127.28.82.194:46645): tablet 55b4f66b9fdf47a79989495eac56488c (table test-table [id=29e8e1961a854553b3d4864ea1f920cd]) successfully deleted
I20260807 22:39:31.810521 23457 ts_tablet_manager.cc:1905] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:31.811813 23457 ts_tablet_manager.cc:1918] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:31.811941 23457 log.cc:1196] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/55b4f66b9fdf47a79989495eac56488c
I20260807 22:39:31.812286 23457 ts_tablet_manager.cc:1939] T 55b4f66b9fdf47a79989495eac56488c P 06012df9a46f450ab44dfde6e04f915b: Deleting consensus metadata
I20260807 22:39:31.812731 22652 catalog_manager.cc:4841] TS 06012df9a46f450ab44dfde6e04f915b (127.28.82.195:38461): tablet 55b4f66b9fdf47a79989495eac56488c (table test-table [id=29e8e1961a854553b3d4864ea1f920cd]) successfully deleted
W20260807 22:39:31.817317 22651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:37801 (error 108)
I20260807 22:39:31.817545 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 22984
I20260807 22:39:31.822180 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.822026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:37801 (local address 127.0.0.1:57682)
0807 22:39:31.822094 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:31.822141 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:37801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 22:39:31.822337 22651 catalog_manager.cc:4584] TS b1d595901e7c4e07a6acc7991b177469 (127.28.82.193:37801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55b4f66b9fdf47a79989495eac56488c: Network error: Client connection negotiation failed: client connection to 127.28.82.193:37801: connect: Connection refused (error 111)
I20260807 22:39:31.827847 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 23193
I20260807 22:39:31.836877 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 22629
2026-08-07T22:39:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2144 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T22:39:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:31Z Disabled control of system clock
I20260807 22:39:31.868038 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37793
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:37793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:31.949730 23468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:31.949951 23468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:31.949972 23468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:31.951085 23468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:31.951121 23468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:31.951135 23468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:31.951148 23468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:31.951160 23468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:31.952421 23468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:37793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:31.952622 23468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:31.952860 23468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:31.957607 23474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:31.957623 23468 server_base.cc:1034] running on GCE node
W20260807 22:39:31.957670 23475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:31.957608 23477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:31.958140 23468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:31.958510 23468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:31.959666 23468 hybrid_clock.cc:648] HybridClock initialized: now 1786142371959651 us; error 32 us; skew 500 ppm
I20260807 22:39:31.960860 23468 webserver.cc:460] Webserver started at http://127.28.82.254:38721/ using document root <none> and password file <none>
I20260807 22:39:31.961112 23468 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:31.961182 23468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:31.961349 23468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:31.962419 23468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "e244033ef7a625ec4716477121795b3d"
server_key_iv: "100ed9eff1cc3acccd5f8866a8d1c0b3"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.962852 23468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "e244033ef7a625ec4716477121795b3d"
server_key_iv: "100ed9eff1cc3acccd5f8866a8d1c0b3"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.964148 23468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:31.964943 23487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:31.965122 23468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:31.965229 23468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3"
format_stamp: "Formatted at 2026-08-07 22:39:31 on dist-test-slave-7kzw"
server_key: "e244033ef7a625ec4716477121795b3d"
server_key_iv: "100ed9eff1cc3acccd5f8866a8d1c0b3"
server_key_version: "encryptionkey@0"
I20260807 22:39:31.965310 23468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:31.983387 23468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:31.983719 23468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:31.983877 23468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:31.987941 23468 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:37793
I20260807 22:39:31.987982 23576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:37793 every 8 connection(s)
I20260807 22:39:31.988354 23468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:31.989356 23577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:31.991775 23577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3: Bootstrap starting.
I20260807 22:39:31.992362 23577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:31.992429 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 23468
I20260807 22:39:31.992538 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:31.992727 23577 log.cc:824] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:31.992756 29003 external_mini_cluster.cc:1428] Setting key c86e2914dd8c0fc66d3c6d5b0b537117
I20260807 22:39:31.993556 23577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3: No bootstrap required, opened a new log
I20260807 22:39:31.995645 23577 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37793 } }
I20260807 22:39:31.995801 23577 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:31.995852 23577 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b4e8396f504ee8bd43fc4d7c6f38f3, State: Initialized, Role: FOLLOWER
I20260807 22:39:31.995986 23577 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37793 } }
I20260807 22:39:31.996068 23577 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:31.996112 23577 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:31.996161 23577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:31.996672 23577 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37793 } }
I20260807 22:39:31.996795 23577 leader_election.cc:302] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0b4e8396f504ee8bd43fc4d7c6f38f3; no voters:
I20260807 22:39:31.996999 23577 leader_election.cc:288] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:31.997057 23584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:31.997201 23584 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [term 1 LEADER]: Becoming Leader. State: Replica: b0b4e8396f504ee8bd43fc4d7c6f38f3, State: Running, Role: LEADER
I20260807 22:39:31.997303 23577 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:31.997318 23584 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37793 } }
I20260807 22:39:31.997818 23586 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37793 } } }
I20260807 22:39:31.997934 23586 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:31.998260 23595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:31.997702 23585 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0b4e8396f504ee8bd43fc4d7c6f38f3. Latest consensus state: current_term: 1 leader_uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0b4e8396f504ee8bd43fc4d7c6f38f3" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37793 } } }
I20260807 22:39:31.998661 23585 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:31.998844 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:31.993280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.28.82.254:37793)
0807 22:39:31.993503 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:31.993512 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:31.993550 (+ 38us) server_negotiation.cc:407] Connection header received
0807 22:39:31.994008 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:31.994014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:31.994080 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:31.994223 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:31.994453 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.995377 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.996414 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:31.996870 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:31.997046 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:31.998030 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:31.998034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:31.998034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:31.998255 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:31.998297 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:31.998312 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:31.998376 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:31.998525 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 22:39:31.998620 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260807 22:39:31.999053 23595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:32.000645 23595 catalog_manager.cc:1347] Generated new cluster ID: dc754274ddac4bb6951b549e85b7d20e
I20260807 22:39:32.000710 23595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:32.029060 23595 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:32.029708 23595 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:32.036002 23595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3: Generated new TSK 0
I20260807 22:39:32.036185 23595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:32.037703 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37793
--builtin_ntp_servers=127.28.82.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:32.121594 23610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:32.121834 23610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:32.121872 23610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:32.123064 23610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:32.123123 23610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:32.123162 23610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:39:32.124501 23610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:32.124763 23610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:32.125057 23610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:32.131026 23621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:32.131057 23619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:32.131235 23610 server_base.cc:1034] running on GCE node
W20260807 22:39:32.131290 23617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:32.131557 23610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:32.131825 23610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:32.133019 23610 hybrid_clock.cc:648] HybridClock initialized: now 1786142372133001 us; error 32 us; skew 500 ppm
I20260807 22:39:32.134681 23610 webserver.cc:460] Webserver started at http://127.28.82.193:40983/ using document root <none> and password file <none>
I20260807 22:39:32.134940 23610 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:32.134995 23610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:32.135183 23610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:32.136312 23610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "ec4a8b97a8f744368e377bbdc133271e"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "f46a0a50bf63d796c9b5ed33931420ad"
server_key_iv: "d5af0153a0205cc154b29868730156fc"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.136714 23610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "ec4a8b97a8f744368e377bbdc133271e"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "f46a0a50bf63d796c9b5ed33931420ad"
server_key_iv: "d5af0153a0205cc154b29868730156fc"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.138156 23610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:32.139063 23629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.139247 23610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.139356 23610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "ec4a8b97a8f744368e377bbdc133271e"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "f46a0a50bf63d796c9b5ed33931420ad"
server_key_iv: "d5af0153a0205cc154b29868730156fc"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.139442 23610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:32.150388 23610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:32.150732 23610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:32.150888 23610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:32.151116 23610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:32.151435 23610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:32.151491 23610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.151531 23610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:32.151556 23610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.157608 23610 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:43715
I20260807 22:39:32.157653 23797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:43715 every 8 connection(s)
I20260807 22:39:32.157966 23610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:39:32.162926 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.158711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:41845 (local address 127.28.82.254:37793)
0807 22:39:32.158759 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.158762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.159030 (+ 268us) server_negotiation.cc:407] Connection header received
0807 22:39:32.159479 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.159480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.159491 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.159538 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:32.160190 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.160671 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.161645 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.161832 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.161883 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.162737 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:32.162738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:32.162738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:32.162742 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:32.162752 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:32.162755 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:32.162796 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.162849 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.162867 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:39:32.163136 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.158710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37793 (local address 127.28.82.193:41845)
0807 22:39:32.158908 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.158941 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.159399 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.159601 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.159604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.159611 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:32.160121 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.160125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.160753 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.160753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.161565 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.161568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.161905 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.161906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.162011 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.162614 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:32.162615 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:32.162643 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:32.162665 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:32.162796 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:32.162798 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:32.162800 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.162843 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.162917 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":92,"threads_started":1}
I20260807 22:39:32.163733 23798 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37793
I20260807 22:39:32.163813 23798 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:32.163815 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 23610
I20260807 22:39:32.163911 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:39:32.164017 23798 heartbeater.cc:511] Master 127.28.82.254:37793 requested a full tablet report, sending...
I20260807 22:39:32.164153 29003 external_mini_cluster.cc:1428] Setting key de40207a9549fdbce39fc719b93e0a87
I20260807 22:39:32.164621 23513 ts_manager.cc:194] Registered new tserver with Master: ec4a8b97a8f744368e377bbdc133271e (127.28.82.193:43715)
I20260807 22:39:32.165280 23513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.193:41845
I20260807 22:39:32.165302 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37793
--builtin_ntp_servers=127.28.82.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:32.248198 23803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:32.248431 23803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:32.248467 23803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:32.249595 23803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:32.249658 23803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:32.249697 23803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:39:32.251092 23803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:32.251354 23803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:32.251653 23803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:32.256669 23812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:32.256680 23816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:32.256675 23803 server_base.cc:1034] running on GCE node
W20260807 22:39:32.256695 23810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:32.257198 23803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:32.257391 23803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:32.258531 23803 hybrid_clock.cc:648] HybridClock initialized: now 1786142372258517 us; error 29 us; skew 500 ppm
I20260807 22:39:32.259789 23803 webserver.cc:460] Webserver started at http://127.28.82.194:34525/ using document root <none> and password file <none>
I20260807 22:39:32.260047 23803 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:32.260093 23803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:32.260279 23803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:32.261300 23803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "4b560c81b88244aca4fcc967c5fb8b08"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "8ec088f849aa646265a3e8c25432c267"
server_key_iv: "4c58cd2b5ace913c0538b745e9c35e2c"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.261651 23803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "4b560c81b88244aca4fcc967c5fb8b08"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "8ec088f849aa646265a3e8c25432c267"
server_key_iv: "4c58cd2b5ace913c0538b745e9c35e2c"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.263073 23803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:32.263862 23824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.264065 23803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:32.264173 23803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "4b560c81b88244aca4fcc967c5fb8b08"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "8ec088f849aa646265a3e8c25432c267"
server_key_iv: "4c58cd2b5ace913c0538b745e9c35e2c"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.264254 23803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:32.275476 23803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:32.275815 23803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:32.275928 23803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:32.276114 23803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:32.276376 23803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:32.276407 23803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.276429 23803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:32.276441 23803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.282687 23803 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:40735
I20260807 22:39:32.282799 23997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:40735 every 8 connection(s)
I20260807 22:39:32.283143 23803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:39:32.288167 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.283916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:54355 (local address 127.28.82.254:37793)
0807 22:39:32.283965 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.283967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.284233 (+ 266us) server_negotiation.cc:407] Connection header received
0807 22:39:32.284694 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.284695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.284706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.284770 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:32.285404 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.285885 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.286894 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.287079 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.287133 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.287919 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:32.287920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:32.287920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:32.287924 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:32.287937 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:32.287941 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:32.287984 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.288069 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.288090 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 22:39:32.288359 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.283905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37793 (local address 127.28.82.194:54355)
0807 22:39:32.284106 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.284138 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.284615 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.284821 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.284824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.284832 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:32.285336 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.285340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.285971 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.285972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.286816 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.286819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.287161 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.287162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.287280 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.287786 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:32.287786 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:32.287813 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:32.287836 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:32.287997 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:32.288001 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:32.288003 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.288056 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.288133 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260807 22:39:32.288870 23998 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37793
I20260807 22:39:32.288975 23998 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:32.289149 23998 heartbeater.cc:511] Master 127.28.82.254:37793 requested a full tablet report, sending...
I20260807 22:39:32.289508 23513 ts_manager.cc:194] Registered new tserver with Master: 4b560c81b88244aca4fcc967c5fb8b08 (127.28.82.194:40735)
I20260807 22:39:32.289856 23513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.194:54355
I20260807 22:39:32.291026 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 23803
I20260807 22:39:32.291134 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:39:32.291343 29003 external_mini_cluster.cc:1428] Setting key a4eaa2d263804e484f89c2e87e18e84d
I20260807 22:39:32.292228 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37793
--builtin_ntp_servers=127.28.82.212:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:32.376040 24001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:32.376241 24001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:32.376263 24001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:32.377372 24001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:32.377413 24001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:32.377465 24001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:39:32.378844 24001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:32.379029 24001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:32.379276 24001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:32.384223 24009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:32.384266 24008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:32.384207 24013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:32.384394 24001 server_base.cc:1034] running on GCE node
I20260807 22:39:32.384609 24001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:32.384863 24001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:32.386029 24001 hybrid_clock.cc:648] HybridClock initialized: now 1786142372386014 us; error 42 us; skew 500 ppm
I20260807 22:39:32.387331 24001 webserver.cc:460] Webserver started at http://127.28.82.195:38231/ using document root <none> and password file <none>
I20260807 22:39:32.387591 24001 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:32.387673 24001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:32.387841 24001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:32.388936 24001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "929220ec0c4c4742a322ede86bd92ab1"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "0d4a3418fe17dd11ea8bf6a5990b5c67"
server_key_iv: "5a40e915ffd900a79e6e4fce4623861e"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.389323 24001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "929220ec0c4c4742a322ede86bd92ab1"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "0d4a3418fe17dd11ea8bf6a5990b5c67"
server_key_iv: "5a40e915ffd900a79e6e4fce4623861e"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.390688 24001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:32.391428 24025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.391589 24001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:32.391700 24001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "929220ec0c4c4742a322ede86bd92ab1"
format_stamp: "Formatted at 2026-08-07 22:39:32 on dist-test-slave-7kzw"
server_key: "0d4a3418fe17dd11ea8bf6a5990b5c67"
server_key_iv: "5a40e915ffd900a79e6e4fce4623861e"
server_key_version: "encryptionkey@0"
I20260807 22:39:32.391775 24001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:32.401502 24001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:32.401775 24001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:32.401925 24001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:32.402144 24001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:32.402454 24001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:32.402577 24001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.402609 24001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:32.402639 24001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:32.408695 24001 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:43295
I20260807 22:39:32.408776 24188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:43295 every 8 connection(s)
I20260807 22:39:32.409092 24001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:39:32.414077 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.409874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:45633 (local address 127.28.82.254:37793)
0807 22:39:32.409925 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.409927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.410170 (+ 243us) server_negotiation.cc:407] Connection header received
0807 22:39:32.410707 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.410708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.410718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.410779 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:32.411429 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.411909 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.412849 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.413030 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.413081 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.413839 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:32.413840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:32.413841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:32.413845 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:32.413859 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:32.413863 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:32.413906 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.413996 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.414017 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:39:32.414250 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.409830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37793 (local address 127.28.82.195:45633)
0807 22:39:32.410038 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.410077 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.410617 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.410857 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.410860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.410868 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:32.411352 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.411356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.411994 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.411995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.412770 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.412772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.413111 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.413112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.413226 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.413704 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:32.413705 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:32.413732 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:32.413753 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:32.413921 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:32.413926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:32.413928 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.413978 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.414054 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
I20260807 22:39:32.414814 24189 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37793
I20260807 22:39:32.414894 24189 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:32.415033 24189 heartbeater.cc:511] Master 127.28.82.254:37793 requested a full tablet report, sending...
I20260807 22:39:32.415346 23513 ts_manager.cc:194] Registered new tserver with Master: 929220ec0c4c4742a322ede86bd92ab1 (127.28.82.195:43295)
I20260807 22:39:32.415702 23513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.195:45633
I20260807 22:39:32.418166 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 24001
I20260807 22:39:32.418255 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:39:32.418529 29003 external_mini_cluster.cc:1428] Setting key 27601e32d43df73bc0a1dc8fb321764d
I20260807 22:39:32.419523 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:39:32.423738 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.420970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.28.82.254:37793)
0807 22:39:32.421027 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.421029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.421166 (+ 137us) server_negotiation.cc:407] Connection header received
0807 22:39:32.421233 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.421234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.421241 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.421290 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:32.421463 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.421910 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.422696 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.422871 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.422923 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.423517 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:32.423518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:32.423518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:32.423522 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:32.423535 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:32.423538 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:32.423613 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.423675 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.423684 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:39:32.425400 23513 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:39:32.427037 23513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:39:32.433521 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.429295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:40735 (local address 127.0.0.1:35414)
0807 22:39:32.429457 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.429500 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.429567 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.429973 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.429975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.429980 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:32.430083 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.430086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.431126 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.431127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.432228 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.432231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.433197 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.433198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.433282 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.433295 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.433365 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.433422 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":4,"thread_start_us":110,"threads_started":1}
I20260807 22:39:32.433548 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.429476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.28.82.194:40735)
0807 22:39:32.429802 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.429806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.429817 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:39:32.429856 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.429858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.429874 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.429949 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:32.430178 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.431041 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.432330 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.433093 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.433168 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.433262 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.433393 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.433438 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":94,"threads_started":1}
I20260807 22:39:32.433840 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.429562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.28.82.193:43715)
0807 22:39:32.429743 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.429748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.430210 (+ 462us) server_negotiation.cc:407] Connection header received
0807 22:39:32.430257 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.430259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.430277 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.430358 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:32.430508 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.431572 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.432680 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.433200 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.433269 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.433361 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.433568 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.433761 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260807 22:39:32.433993 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.429297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:43295 (local address 127.0.0.1:37098)
0807 22:39:32.429677 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.429689 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.429734 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.429972 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.429975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.429977 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:32.430083 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.430085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.431596 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.431597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.432651 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.432653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.433659 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.433660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.433728 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.433733 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.433790 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.433829 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":25,"thread_start_us":66,"threads_started":1}
I20260807 22:39:32.434041 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.429527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.28.82.195:43295)
0807 22:39:32.429760 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.429765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.429780 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:39:32.429821 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.429823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.429839 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.429933 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:32.430354 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.431494 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.432741 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.433468 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.433533 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.433617 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.433925 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.433968 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":115,"threads_started":1}
I20260807 22:39:32.434425 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.429297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43715 (local address 127.0.0.1:45162)
0807 22:39:32.430078 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.430089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.430149 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.430372 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.430373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.430375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:32.430410 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.430411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.431638 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.431639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.432587 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.432590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.433396 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.433397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.433446 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.433449 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.433496 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.433527 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260807 22:39:32.434288 23896 tablet_service.cc:1467] Processing CreateTablet for tablet 7dd351a3ccad4fb6b874acf958711ca4 (DEFAULT_TABLE table=test-table [id=6488fbcb9f794fc78c2b34a3293e175c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:32.434716 23896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dd351a3ccad4fb6b874acf958711ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:32.434737 24094 tablet_service.cc:1467] Processing CreateTablet for tablet 7dd351a3ccad4fb6b874acf958711ca4 (DEFAULT_TABLE table=test-table [id=6488fbcb9f794fc78c2b34a3293e175c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:32.435067 24094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dd351a3ccad4fb6b874acf958711ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:32.435156 23697 tablet_service.cc:1467] Processing CreateTablet for tablet 7dd351a3ccad4fb6b874acf958711ca4 (DEFAULT_TABLE table=test-table [id=6488fbcb9f794fc78c2b34a3293e175c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:32.435407 23697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dd351a3ccad4fb6b874acf958711ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:32.438211 24215 tablet_bootstrap.cc:492] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e: Bootstrap starting.
I20260807 22:39:32.438328 24218 tablet_bootstrap.cc:492] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Bootstrap starting.
I20260807 22:39:32.438943 24218 tablet_bootstrap.cc:654] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:32.438946 24215 tablet_bootstrap.cc:654] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:32.439270 24215 log.cc:824] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:32.439270 24218 log.cc:824] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:32.439639 24216 tablet_bootstrap.cc:492] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Bootstrap starting.
I20260807 22:39:32.440045 24215 tablet_bootstrap.cc:492] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e: No bootstrap required, opened a new log
I20260807 22:39:32.440102 24215 ts_tablet_manager.cc:1397] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:39:32.440228 24216 tablet_bootstrap.cc:654] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:32.440523 24216 log.cc:824] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:32.441075 24218 tablet_bootstrap.cc:492] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: No bootstrap required, opened a new log
I20260807 22:39:32.441145 24218 ts_tablet_manager.cc:1397] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 22:39:32.442292 24215 raft_consensus.cc:348] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.442436 24215 raft_consensus.cc:374] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:32.442502 24215 raft_consensus.cc:729] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec4a8b97a8f744368e377bbdc133271e, State: Initialized, Role: FOLLOWER
I20260807 22:39:32.442704 24215 consensus_queue.cc:260] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.442981 24215 ts_tablet_manager.cc:1428] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:39:32.443055 23798 heartbeater.cc:503] Master 127.28.82.254:37793 was elected leader, sending a full tablet report...
I20260807 22:39:32.443547 24218 raft_consensus.cc:348] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.443691 24218 raft_consensus.cc:374] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:32.443737 24218 raft_consensus.cc:729] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b560c81b88244aca4fcc967c5fb8b08, State: Initialized, Role: FOLLOWER
I20260807 22:39:32.443861 24218 consensus_queue.cc:260] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.444142 24218 ts_tablet_manager.cc:1428] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:39:32.444175 23998 heartbeater.cc:503] Master 127.28.82.254:37793 was elected leader, sending a full tablet report...
I20260807 22:39:32.444542 24216 tablet_bootstrap.cc:492] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: No bootstrap required, opened a new log
I20260807 22:39:32.444633 24216 ts_tablet_manager.cc:1397] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 22:39:32.446780 24216 raft_consensus.cc:348] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.446902 24216 raft_consensus.cc:374] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:32.446923 24216 raft_consensus.cc:729] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929220ec0c4c4742a322ede86bd92ab1, State: Initialized, Role: FOLLOWER
I20260807 22:39:32.446997 24216 consensus_queue.cc:260] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.447234 24216 ts_tablet_manager.cc:1428] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 22:39:32.447290 24189 heartbeater.cc:503] Master 127.28.82.254:37793 was elected leader, sending a full tablet report...
W20260807 22:39:32.533758 23999 tablet.cc:2369] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:39:32.658730 23799 tablet.cc:2369] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 22:39:32.659689 24192 tablet.cc:2369] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:39:32.703836 24228 raft_consensus.cc:482] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:32.703997 24228 raft_consensus.cc:504] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.704490 24228 leader_election.cc:288] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec4a8b97a8f744368e377bbdc133271e (127.28.82.193:43715), 929220ec0c4c4742a322ede86bd92ab1 (127.28.82.195:43295)
I20260807 22:39:32.708771 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.704707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43715 (local address 127.28.82.194:52891)
0807 22:39:32.704780 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.704794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.704845 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.705050 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.705050 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.705052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:32.705157 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.705159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.706117 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.706118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.707536 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.707542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.708423 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.708425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.708531 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.708535 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.708601 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.708652 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":80}
I20260807 22:39:32.708792 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.704791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:57387 (local address 127.28.82.195:43295)
0807 22:39:32.704839 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.704841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.704979 (+ 138us) server_negotiation.cc:407] Connection header received
0807 22:39:32.705012 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.705013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.705023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.705082 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:32.705257 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.706018 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.707735 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.708326 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.708408 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.708417 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.708664 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.708717 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 22:39:32.708792 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.704791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:52891 (local address 127.28.82.193:43715)
0807 22:39:32.704839 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.704841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.704854 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:39:32.704919 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.704920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.704929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.705027 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:32.705251 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.706020 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.707735 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.708323 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.708408 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.708417 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.708664 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.708717 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:39:32.708779 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.704706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:43295 (local address 127.28.82.194:57387)
0807 22:39:32.704899 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.704907 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.704962 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.705093 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.705094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.705095 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:32.705157 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.705159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.706117 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.706118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.707541 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.707543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.708424 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.708425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.708531 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.708535 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.708601 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.708652 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":116,"thread_start_us":114,"threads_started":1}
I20260807 22:39:32.709131 24229 raft_consensus.cc:482] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:32.709254 24229 raft_consensus.cc:504] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.709291 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd351a3ccad4fb6b874acf958711ca4" candidate_uuid: "4b560c81b88244aca4fcc967c5fb8b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec4a8b97a8f744368e377bbdc133271e" is_pre_election: true
I20260807 22:39:32.709331 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd351a3ccad4fb6b874acf958711ca4" candidate_uuid: "4b560c81b88244aca4fcc967c5fb8b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929220ec0c4c4742a322ede86bd92ab1" is_pre_election: true
I20260807 22:39:32.709424 23732 raft_consensus.cc:2454] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b560c81b88244aca4fcc967c5fb8b08 in term 0.
I20260807 22:39:32.709444 24123 raft_consensus.cc:2454] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b560c81b88244aca4fcc967c5fb8b08 in term 0.
I20260807 22:39:32.709738 23831 leader_election.cc:302] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b560c81b88244aca4fcc967c5fb8b08, ec4a8b97a8f744368e377bbdc133271e; no voters:
I20260807 22:39:32.709911 24228 raft_consensus.cc:2781] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:39:32.709975 24228 raft_consensus.cc:482] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:39:32.710004 24228 raft_consensus.cc:3037] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:32.710068 24229 leader_election.cc:288] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec4a8b97a8f744368e377bbdc133271e (127.28.82.193:43715), 4b560c81b88244aca4fcc967c5fb8b08 (127.28.82.194:40735)
I20260807 22:39:32.711033 24228 raft_consensus.cc:504] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.711176 24228 leader_election.cc:288] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [CANDIDATE]: Term 1 election: Requested vote from peers ec4a8b97a8f744368e377bbdc133271e (127.28.82.193:43715), 929220ec0c4c4742a322ede86bd92ab1 (127.28.82.195:43295)
I20260807 22:39:32.711376 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd351a3ccad4fb6b874acf958711ca4" candidate_uuid: "4b560c81b88244aca4fcc967c5fb8b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929220ec0c4c4742a322ede86bd92ab1"
I20260807 22:39:32.711452 24123 raft_consensus.cc:3037] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:32.711423 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd351a3ccad4fb6b874acf958711ca4" candidate_uuid: "4b560c81b88244aca4fcc967c5fb8b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec4a8b97a8f744368e377bbdc133271e"
I20260807 22:39:32.711508 23732 raft_consensus.cc:3037] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:32.712270 23732 raft_consensus.cc:2454] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b560c81b88244aca4fcc967c5fb8b08 in term 1.
I20260807 22:39:32.712330 24123 raft_consensus.cc:2454] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b560c81b88244aca4fcc967c5fb8b08 in term 1.
I20260807 22:39:32.712469 23831 leader_election.cc:302] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b560c81b88244aca4fcc967c5fb8b08, ec4a8b97a8f744368e377bbdc133271e; no voters:
I20260807 22:39:32.712566 24228 raft_consensus.cc:2781] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:32.712771 24228 raft_consensus.cc:686] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 1 LEADER]: Becoming Leader. State: Replica: 4b560c81b88244aca4fcc967c5fb8b08, State: Running, Role: LEADER
I20260807 22:39:32.712874 24228 consensus_queue.cc:237] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } }
I20260807 22:39:32.713459 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.709948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:48705 (local address 127.28.82.194:40735)
0807 22:39:32.709991 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.709993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.710001 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:39:32.710025 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.710025 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.710044 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.710102 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:32.710297 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.710981 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.712365 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.713100 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.713194 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.713203 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.713359 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.713395 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:39:32.713462 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.709761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:40735 (local address 127.28.82.195:48705)
0807 22:39:32.709844 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.709855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.709895 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.710150 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.710151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.710153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:32.710218 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.710219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.711092 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.711093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.712260 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.712263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.713205 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.713207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.713283 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.713286 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.713353 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.713397 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":3}
I20260807 22:39:32.713786 23922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd351a3ccad4fb6b874acf958711ca4" candidate_uuid: "929220ec0c4c4742a322ede86bd92ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b560c81b88244aca4fcc967c5fb8b08" is_pre_election: true
I20260807 22:39:32.713858 23513 catalog_manager.cc:5495] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b560c81b88244aca4fcc967c5fb8b08 (127.28.82.194). New cstate: current_term: 1 leader_uuid: "4b560c81b88244aca4fcc967c5fb8b08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b560c81b88244aca4fcc967c5fb8b08" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 40735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 } health_report { overall_health: UNKNOWN } } }
I20260807 22:39:32.714883 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.709761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43715 (local address 127.28.82.195:54155)
0807 22:39:32.710186 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.710195 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:32.710269 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:32.710403 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:32.710404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:32.710405 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:32.710502 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.710504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.711259 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.711259 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.713417 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:32.713420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.714624 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.714626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:32.714701 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.714704 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:32.714763 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:39:32.714805 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":1262,"thread_start_us":96,"threads_started":1}
I20260807 22:39:32.714928 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.709904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:54155 (local address 127.28.82.193:43715)
0807 22:39:32.709940 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.709942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.710276 (+ 334us) server_negotiation.cc:407] Connection header received
0807 22:39:32.710324 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.710325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.710334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.710388 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:32.710584 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.711164 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.714000 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.714522 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.714624 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.714632 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.714806 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.714851 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:39:32.715288 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd351a3ccad4fb6b874acf958711ca4" candidate_uuid: "929220ec0c4c4742a322ede86bd92ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec4a8b97a8f744368e377bbdc133271e" is_pre_election: true
I20260807 22:39:32.715364 23732 raft_consensus.cc:2379] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 929220ec0c4c4742a322ede86bd92ab1 in current term 1: Already voted for candidate 4b560c81b88244aca4fcc967c5fb8b08 in this term.
I20260807 22:39:32.715517 24029 leader_election.cc:302] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 929220ec0c4c4742a322ede86bd92ab1; no voters: 4b560c81b88244aca4fcc967c5fb8b08, ec4a8b97a8f744368e377bbdc133271e
I20260807 22:39:32.715633 24229 raft_consensus.cc:2726] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 22:39:32.786170 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.783580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.28.82.194:40735)
0807 22:39:32.783649 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:32.783651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:32.783660 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:39:32.783704 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:32.783705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:32.783734 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:32.783825 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:39:32.783972 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.784552 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.785324 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:32.785532 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:32.785609 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:32.785765 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:39:32.785870 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:39:32.785992 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:32.786045 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 22:39:32.786056 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:39:32.788002 23732 raft_consensus.cc:1261] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 1 FOLLOWER]: Refusing update from remote peer 4b560c81b88244aca4fcc967c5fb8b08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:32.788043 24123 raft_consensus.cc:1261] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 1 FOLLOWER]: Refusing update from remote peer 4b560c81b88244aca4fcc967c5fb8b08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:32.788411 24228 consensus_queue.cc:1035] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929220ec0c4c4742a322ede86bd92ab1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 43295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:32.788573 24232 consensus_queue.cc:1035] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec4a8b97a8f744368e377bbdc133271e" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 43715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:32.791108 24237 mvcc.cc:205] Tried to move back new op lower bound from 7316039158937042944 to 7316039158632423424. Current Snapshot: MvccSnapshot[applied={T|T < 7316039158937042944}]
I20260807 22:39:32.791276 24238 mvcc.cc:205] Tried to move back new op lower bound from 7316039158937042944 to 7316039158632423424. Current Snapshot: MvccSnapshot[applied={T|T < 7316039158937042944}]
I20260807 22:39:32.791715 24236 mvcc.cc:205] Tried to move back new op lower bound from 7316039158937042944 to 7316039158632423424. Current Snapshot: MvccSnapshot[applied={T|T < 7316039158937042944}]
I20260807 22:39:32.793859 23513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:32.793962 23513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:32.794785 23513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0b4e8396f504ee8bd43fc4d7c6f38f3: Sending DeleteTablet for 3 replicas of tablet 7dd351a3ccad4fb6b874acf958711ca4
I20260807 22:39:32.795147 23697 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dd351a3ccad4fb6b874acf958711ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:32 UTC) from {username='slave'} at 127.0.0.1:45162
I20260807 22:39:32.795166 23896 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dd351a3ccad4fb6b874acf958711ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:32 UTC) from {username='slave'} at 127.0.0.1:35414
I20260807 22:39:32.795167 24094 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dd351a3ccad4fb6b874acf958711ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:32 UTC) from {username='slave'} at 127.0.0.1:37098
I20260807 22:39:32.795393 24260 tablet_replica.cc:330] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: stopping tablet replica
I20260807 22:39:32.795490 24260 raft_consensus.cc:2229] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:32.795604 24260 raft_consensus.cc:2258] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:32.795763 24258 tablet_replica.cc:330] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e: stopping tablet replica
I20260807 22:39:32.795802 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 23610
I20260807 22:39:32.795852 24258 raft_consensus.cc:2229] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:32.795943 24258 raft_consensus.cc:2258] T 7dd351a3ccad4fb6b874acf958711ca4 P ec4a8b97a8f744368e377bbdc133271e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:32.796093 24260 ts_tablet_manager.cc:1905] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:32.797855 24260 ts_tablet_manager.cc:1918] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:32.797926 24260 log.cc:1196] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/7dd351a3ccad4fb6b874acf958711ca4
I20260807 22:39:32.798194 24260 ts_tablet_manager.cc:1939] T 7dd351a3ccad4fb6b874acf958711ca4 P 929220ec0c4c4742a322ede86bd92ab1: Deleting consensus metadata
I20260807 22:39:32.798661 23491 catalog_manager.cc:4841] TS 929220ec0c4c4742a322ede86bd92ab1 (127.28.82.195:43295): tablet 7dd351a3ccad4fb6b874acf958711ca4 (table test-table [id=6488fbcb9f794fc78c2b34a3293e175c]) successfully deleted
I20260807 22:39:32.803687 24263 tablet_replica.cc:330] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: stopping tablet replica
I20260807 22:39:32.803869 24263 raft_consensus.cc:2229] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:39:32.804047 24263 raft_consensus.cc:2258] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:32.804538 24263 ts_tablet_manager.cc:1905] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:32.806150 24263 ts_tablet_manager.cc:1918] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:32.806229 24263 log.cc:1196] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/7dd351a3ccad4fb6b874acf958711ca4
I20260807 22:39:32.806511 24263 ts_tablet_manager.cc:1939] T 7dd351a3ccad4fb6b874acf958711ca4 P 4b560c81b88244aca4fcc967c5fb8b08: Deleting consensus metadata
I20260807 22:39:32.806938 23493 catalog_manager.cc:4841] TS 4b560c81b88244aca4fcc967c5fb8b08 (127.28.82.194:40735): tablet 7dd351a3ccad4fb6b874acf958711ca4 (table test-table [id=6488fbcb9f794fc78c2b34a3293e175c]) successfully deleted
W20260807 22:39:32.808709 23494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:43715 (error 108)
I20260807 22:39:32.808992 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 23803
I20260807 22:39:32.809201 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:32.809072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:43715 (local address 127.0.0.1:45178)
0807 22:39:32.809122 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:32.809160 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 22:39:32.809309 23494 catalog_manager.cc:4584] TS ec4a8b97a8f744368e377bbdc133271e (127.28.82.193:43715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7dd351a3ccad4fb6b874acf958711ca4: Network error: Client connection negotiation failed: client connection to 127.28.82.193:43715: connect: Connection refused (error 111)
I20260807 22:39:32.818065 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 24001
I20260807 22:39:32.824718 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 23468
2026-08-07T22:39:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (990 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:32 dist-test-slave-7kzw krb5kdc[24268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:32 dist-test-slave-7kzw krb5kdc[24268](info): set up 2 sockets
Aug 07 22:39:32 dist-test-slave-7kzw krb5kdc[24268](info): commencing operation
krb5kdc: starting...
W20260807 22:39:34.031826 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:39:34.159610 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45499
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:45499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:34.242870 24287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:34.243072 24287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:34.243094 24287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:34.244254 24287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:34.244298 24287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:34.244313 24287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:34.244325 24287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:34.244338 24287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:34.245797 24287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:45499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:34.246043 24287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:34.246302 24287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:34.251262 24297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:34.251264 24293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:34.251283 24295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:34.251801 24287 server_base.cc:1034] running on GCE node
I20260807 22:39:34.252064 24287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:34.252351 24287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:34.253515 24287 hybrid_clock.cc:648] HybridClock initialized: now 1786142374253509 us; error 41 us; skew 500 ppm
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:34.256259 24287 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:39:34.256750 24287 webserver.cc:460] Webserver started at http://127.28.82.254:33671/ using document root <none> and password file <none>
I20260807 22:39:34.256975 24287 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:34.257052 24287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:34.257205 24287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:34.258271 24287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "f7d3a64fae304c1698194e8dca9b9014"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "e7948e80ecadd45b494bd37922cd177c"
server_key_iv: "1012ecead8a90c1b255901a935d91fb3"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.258805 24287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "f7d3a64fae304c1698194e8dca9b9014"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "e7948e80ecadd45b494bd37922cd177c"
server_key_iv: "1012ecead8a90c1b255901a935d91fb3"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.260252 24287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:34.261061 24307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.261268 24287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:39:34.261382 24287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "f7d3a64fae304c1698194e8dca9b9014"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "e7948e80ecadd45b494bd37922cd177c"
server_key_iv: "1012ecead8a90c1b255901a935d91fb3"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.261461 24287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:34.281451 24287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:34.283588 24287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:34.283779 24287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:34.288158 24287 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:45499
I20260807 22:39:34.288195 24380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:45499 every 8 connection(s)
I20260807 22:39:34.288544 24287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:34.289436 24382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:34.291745 24382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014: Bootstrap starting.
I20260807 22:39:34.292342 24382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:34.292658 24382 log.cc:824] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:34.293375 24382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014: No bootstrap required, opened a new log
I20260807 22:39:34.295064 24382 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d3a64fae304c1698194e8dca9b9014" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45499 } }
I20260807 22:39:34.295213 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 24287
I20260807 22:39:34.295236 24382 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:34.295289 24382 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d3a64fae304c1698194e8dca9b9014, State: Initialized, Role: FOLLOWER
I20260807 22:39:34.295333 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:34.295450 24382 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d3a64fae304c1698194e8dca9b9014" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45499 } }
I20260807 22:39:34.295536 24382 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:34.295573 29003 external_mini_cluster.cc:1428] Setting key cdbea4aac687fe716361f95308e73d56
I20260807 22:39:34.295581 24382 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:34.295634 24382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:34.296806 24382 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d3a64fae304c1698194e8dca9b9014" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45499 } }
I20260807 22:39:34.296919 24382 leader_election.cc:302] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7d3a64fae304c1698194e8dca9b9014; no voters:
I20260807 22:39:34.297083 24382 leader_election.cc:288] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:34.297129 24390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:34.297289 24390 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [term 1 LEADER]: Becoming Leader. State: Replica: f7d3a64fae304c1698194e8dca9b9014, State: Running, Role: LEADER
I20260807 22:39:34.297354 24382 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:34.297425 24390 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d3a64fae304c1698194e8dca9b9014" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45499 } }
I20260807 22:39:34.297768 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7d3a64fae304c1698194e8dca9b9014. Latest consensus state: current_term: 1 leader_uuid: "f7d3a64fae304c1698194e8dca9b9014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d3a64fae304c1698194e8dca9b9014" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45499 } } }
I20260807 22:39:34.297863 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:34.297781 24391 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7d3a64fae304c1698194e8dca9b9014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7d3a64fae304c1698194e8dca9b9014" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45499 } } }
I20260807 22:39:34.298069 24397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:34.298081 24391 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:34.298593 24397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:34.299913 24414 mvcc.cc:205] Tried to move back new op lower bound from 7316039165130571776 to 7316039165122748416. Current Snapshot: MvccSnapshot[applied={T|T < 7316039165130571776}]
I20260807 22:39:34.300312 24397 catalog_manager.cc:1347] Generated new cluster ID: 1009fad1f9e84d00b26195bfc3fda49c
I20260807 22:39:34.300364 24397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:34.306807 24397 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:34.307322 24397 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:34.308229 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.296089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.28.82.254:45499)
0807 22:39:34.296392 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.296405 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.296447 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:39:34.296580 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.296588 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.296643 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.296793 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:34.297646 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.299100 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.300256 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.300844 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.300926 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.304105 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:34.304109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:34.304109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:34.304122 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:34.306153 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.306605 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.306609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.306609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.306720 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.307008 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.307009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.307010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.307378 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:34.307527 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.307856 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.308025 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":180,"threads_started":1}
I20260807 22:39:34.312894 24397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014: Generated new TSK 0
I20260807 22:39:34.313073 24397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.82.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
Entry for principal kudu/127.28.82.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab.
I20260807 22:39:34.340970 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45499
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:34.432243 24420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:34.432456 24420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:34.432482 24420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:34.433753 24420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:34.433799 24420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:34.433827 24420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:39:34.435442 24420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.193.keytab
--principal=kudu/127.28.82.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:34.435699 24420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:34.435950 24420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:34.441452 24431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:34.441442 24438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:34.441443 24434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:34.441705 24420 server_base.cc:1034] running on GCE node
I20260807 22:39:34.442008 24420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:34.442224 24420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:34.443380 24420 hybrid_clock.cc:648] HybridClock initialized: now 1786142374443364 us; error 29 us; skew 500 ppm
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:34.446241 24420 init.cc:377] Logged in from keytab as kudu/127.28.82.193@KRBTEST.COM (short username kudu)
I20260807 22:39:34.446770 24420 webserver.cc:460] Webserver started at http://127.28.82.193:44877/ using document root <none> and password file <none>
I20260807 22:39:34.447033 24420 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:34.447083 24420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:34.447289 24420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:34.448410 24420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "51b5b38129e446d689be6f2f2315ba04"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "71ef3663fbe381e93da4ba1ad31446fb"
server_key_iv: "677fd5c0fa08f959c268a9ae4e7d35a9"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.448798 24420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "51b5b38129e446d689be6f2f2315ba04"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "71ef3663fbe381e93da4ba1ad31446fb"
server_key_iv: "677fd5c0fa08f959c268a9ae4e7d35a9"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.450253 24420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:34.451130 24449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.451318 24420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:34.451418 24420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "51b5b38129e446d689be6f2f2315ba04"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "71ef3663fbe381e93da4ba1ad31446fb"
server_key_iv: "677fd5c0fa08f959c268a9ae4e7d35a9"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.451498 24420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:34.477233 24420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:34.479815 24420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:34.479996 24420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:34.480265 24420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:34.480625 24420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:34.480664 24420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.480688 24420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:34.480738 24420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.487282 24420 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:42111
I20260807 22:39:34.487339 24621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:42111 every 8 connection(s)
I20260807 22:39:34.487670 24420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.193@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:34.496788 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 24420
I20260807 22:39:34.496910 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:39:34.497025 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.488381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:43865 (local address 127.28.82.254:45499)
0807 22:39:34.488430 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.488433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.488624 (+ 191us) server_negotiation.cc:407] Connection header received
0807 22:39:34.489136 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.489137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.489147 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.489210 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:34.490165 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.490744 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.491705 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.491911 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.491968 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.494095 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:34.494097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:34.494097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:34.494104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:34.495482 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.495890 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.495891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.495892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.495947 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.496228 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.496229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.496230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.496361 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:34.496436 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.496774 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.496889 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:39:34.497138 29003 external_mini_cluster.cc:1428] Setting key 5bc51c49d1c9abc3178e9030f93e6cd1
I20260807 22:39:34.497217 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.488340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45499 (local address 127.28.82.193:43865)
0807 22:39:34.488508 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.488550 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:34.489037 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:34.489286 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:34.489290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:34.489876 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:34.490066 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.490070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.490832 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.490833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.491626 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.491628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.491986 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.491987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.492089 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.492559 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:34.492568 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:34.494004 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:34.495561 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:34.495568 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:34.495570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:34.495818 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:34.496014 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:34.496015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:34.496015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:34.496139 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:34.496435 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:34.496440 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:34.496608 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 22:39:34.496748 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 22:39:34.496944 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260807 22:39:34.497771 24622 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45499
I20260807 22:39:34.497916 24622 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:34.498162 24622 heartbeater.cc:511] Master 127.28.82.254:45499 requested a full tablet report, sending...
I20260807 22:39:34.498767 24334 ts_manager.cc:194] Registered new tserver with Master: 51b5b38129e446d689be6f2f2315ba04 (127.28.82.193:42111)
I20260807 22:39:34.499485 24334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.193@KRBTEST.COM'} at 127.28.82.193:43865
WARNING: no policy specified for kudu/127.28.82.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
Entry for principal kudu/127.28.82.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab.
I20260807 22:39:34.525139 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45499
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:34.608476 24627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:34.608723 24627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:34.608760 24627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:34.609884 24627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:34.609956 24627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:34.609999 24627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:39:34.611490 24627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.194.keytab
--principal=kudu/127.28.82.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:34.611773 24627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:34.612003 24627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:34.617311 24639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:34.617291 24642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:34.617312 24627 server_base.cc:1034] running on GCE node
W20260807 22:39:34.617274 24640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:34.617699 24627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:34.617959 24627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:34.619134 24627 hybrid_clock.cc:648] HybridClock initialized: now 1786142374619129 us; error 41 us; skew 500 ppm
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:34.621889 24627 init.cc:377] Logged in from keytab as kudu/127.28.82.194@KRBTEST.COM (short username kudu)
I20260807 22:39:34.622325 24627 webserver.cc:460] Webserver started at http://127.28.82.194:32931/ using document root <none> and password file <none>
I20260807 22:39:34.622644 24627 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:34.622714 24627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:34.622853 24627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:34.623865 24627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "0dbc5691456c493e82e8744c9733dc7b"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "130a5571f4b93a1db690c72f45a0f7f3"
server_key_iv: "6eea4a1246a5055602bd694f1d17c998"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.624231 24627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "0dbc5691456c493e82e8744c9733dc7b"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "130a5571f4b93a1db690c72f45a0f7f3"
server_key_iv: "6eea4a1246a5055602bd694f1d17c998"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.625588 24627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:34.626380 24655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.626605 24627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:34.626721 24627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "0dbc5691456c493e82e8744c9733dc7b"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "130a5571f4b93a1db690c72f45a0f7f3"
server_key_iv: "6eea4a1246a5055602bd694f1d17c998"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.626807 24627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:34.642202 24627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:34.644359 24627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:34.644536 24627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:34.644765 24627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:34.645074 24627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:34.645136 24627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.645179 24627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:34.645207 24627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.651578 24627 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:42859
I20260807 22:39:34.651684 24833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:42859 every 8 connection(s)
I20260807 22:39:34.651980 24627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.194@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:34.660233 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 24627
I20260807 22:39:34.660372 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:39:34.660599 29003 external_mini_cluster.cc:1428] Setting key 39207f5bde9310379cbaed056f8addd9
I20260807 22:39:34.661674 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.652812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:40791 (local address 127.28.82.254:45499)
0807 22:39:34.652877 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.652879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.653098 (+ 219us) server_negotiation.cc:407] Connection header received
0807 22:39:34.653636 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.653638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.653653 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.653704 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:34.654731 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.655311 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.656338 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.656535 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.656612 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.658805 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:34.658806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:34.658807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:34.658814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:34.660044 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.660543 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.660544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.660545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.660616 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.660931 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.660932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.660933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.661068 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:34.661125 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.661472 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.661559 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:39:34.661762 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.652794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45499 (local address 127.28.82.194:40791)
0807 22:39:34.652983 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.653017 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:34.653524 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:34.653779 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:34.653783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:34.654364 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:34.654635 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.654641 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.655413 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.655414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.656225 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.656228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.656621 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.656622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.656765 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.657258 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:34.657267 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:34.658717 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:34.660127 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:34.660133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:34.660135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:34.660456 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:34.660699 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:34.660700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:34.660701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:34.660853 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:34.661140 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:34.661146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:34.661312 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 22:39:34.661435 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 22:39:34.661582 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260807 22:39:34.662871 24836 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45499
I20260807 22:39:34.662989 24836 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:34.663159 24836 heartbeater.cc:511] Master 127.28.82.254:45499 requested a full tablet report, sending...
I20260807 22:39:34.663560 24334 ts_manager.cc:194] Registered new tserver with Master: 0dbc5691456c493e82e8744c9733dc7b (127.28.82.194:42859)
I20260807 22:39:34.663938 24334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.194@KRBTEST.COM'} at 127.28.82.194:40791
WARNING: no policy specified for kudu/127.28.82.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
Entry for principal kudu/127.28.82.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab.
I20260807 22:39:34.687506 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45499
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:34.772557 24845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:34.772763 24845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:34.772786 24845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:34.773844 24845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:34.773877 24845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:34.773902 24845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:39:34.775269 24845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.195.keytab
--principal=kudu/127.28.82.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:45499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:34.775442 24845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:34.775609 24845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:34.780732 24852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:34.780857 24853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:34.780753 24856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:34.781124 24845 server_base.cc:1034] running on GCE node
I20260807 22:39:34.781352 24845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:34.781602 24845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:34.782788 24845 hybrid_clock.cc:648] HybridClock initialized: now 1786142374782767 us; error 41 us; skew 500 ppm
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:34.785522 24845 init.cc:377] Logged in from keytab as kudu/127.28.82.195@KRBTEST.COM (short username kudu)
I20260807 22:39:34.786044 24845 webserver.cc:460] Webserver started at http://127.28.82.195:38201/ using document root <none> and password file <none>
I20260807 22:39:34.786298 24845 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:34.786370 24845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:34.786545 24845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:34.787575 24845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "2f632c33740d408ca6941837b5dcf5e1"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "21cace679989f668a430c283df61f74b"
server_key_iv: "240000ab70f64fb99449d4d55df10398"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.787953 24845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "2f632c33740d408ca6941837b5dcf5e1"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "21cace679989f668a430c283df61f74b"
server_key_iv: "240000ab70f64fb99449d4d55df10398"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.789315 24845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:34.790081 24869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.790266 24845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:34.790361 24845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "2f632c33740d408ca6941837b5dcf5e1"
format_stamp: "Formatted at 2026-08-07 22:39:34 on dist-test-slave-7kzw"
server_key: "21cace679989f668a430c283df61f74b"
server_key_iv: "240000ab70f64fb99449d4d55df10398"
server_key_version: "encryptionkey@0"
I20260807 22:39:34.790445 24845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:34.804268 24845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:34.806380 24845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:34.806572 24845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:34.806830 24845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:34.807152 24845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:34.807209 24845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.807253 24845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:34.807302 24845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:34.813954 24845 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:46041
I20260807 22:39:34.814000 25054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:46041 every 8 connection(s)
I20260807 22:39:34.814328 24845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
Aug 07 22:39:34 dist-test-slave-7kzw krb5kdc[24268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142374, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.195@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:34.823144 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 24845
I20260807 22:39:34.823263 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:39:34.823483 29003 external_mini_cluster.cc:1428] Setting key 0be0e44db3a3dc428e1ae8a9f54bdd61
I20260807 22:39:34.823971 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.815095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:46363 (local address 127.28.82.254:45499)
0807 22:39:34.815164 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.815166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.815390 (+ 224us) server_negotiation.cc:407] Connection header received
0807 22:39:34.815936 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.815938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.815950 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.815989 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:34.816941 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.817449 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.818503 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.818718 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.818794 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.820957 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:34.820958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:34.820959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:34.820965 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:34.822255 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.822743 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.822744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.822744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.822800 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.823132 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.823134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.823134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.823304 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:34.823402 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.823747 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.823862 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 22:39:34.824162 25059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.815070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45499 (local address 127.28.82.195:46363)
0807 22:39:34.815273 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.815308 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:34.815830 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:34.816064 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:34.816068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:34.816645 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:34.816840 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.816845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.817560 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.817561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.818381 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.818383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.818815 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.818817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.818939 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.819375 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:39:34.819384 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:34.820859 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:34.822331 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:34.822338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:34.822339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:34.822674 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:34.822890 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:39:34.822891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:39:34.822892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:39:34.823011 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:39:34.823396 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:34.823405 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:34.823580 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 22:39:34.823699 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 22:39:34.823907 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":107,"threads_started":1}
I20260807 22:39:34.824745 25055 heartbeater.cc:347] Connected to a master server at 127.28.82.254:45499
I20260807 22:39:34.824826 25055 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:34.824961 25055 heartbeater.cc:511] Master 127.28.82.254:45499 requested a full tablet report, sending...
I20260807 22:39:34.825300 24334 ts_manager.cc:194] Registered new tserver with Master: 2f632c33740d408ca6941837b5dcf5e1 (127.28.82.195:46041)
I20260807 22:39:34.825712 24334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.82.195@KRBTEST.COM'} at 127.28.82.195:46363
I20260807 22:39:34.835412 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:39:34.843144 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.836798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40964 (local address 127.28.82.254:45499)
0807 22:39:34.836860 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.836862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.836919 (+ 57us) server_negotiation.cc:407] Connection header received
0807 22:39:34.836982 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.836982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.836990 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.837047 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:34.837580 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.838088 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.838883 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.839069 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.839131 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.840348 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:34.840349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:34.840350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:34.840356 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:34.842036 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.842362 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.842363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.842363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.842425 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:34.842705 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:34.842705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:34.842706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:34.842852 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:34.842933 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.843018 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.843078 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:39:34.844659 24334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:39:34.846230 24334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:39:34.852845 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.848536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:42859 (local address 127.0.0.1:34698)
0807 22:39:34.848858 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.848903 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:34.848974 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:34.849204 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:34.849208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:34.849325 (+ 117us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:34.849432 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.849435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.850336 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.850338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.851807 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.851809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.852531 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.852532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.852623 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.852637 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:39:34.852706 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 22:39:34.852762 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":4,"thread_start_us":123,"threads_started":1}
I20260807 22:39:34.853049 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.848879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.28.82.195:46041)
0807 22:39:34.849082 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.849087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.849100 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:39:34.849152 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.849155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.849169 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.849238 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:34.849501 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.850664 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.851951 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.852615 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.852678 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.852733 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.852818 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.852856 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
I20260807 22:39:34.852921 25072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.848728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:46041 (local address 127.0.0.1:60276)
0807 22:39:34.849023 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.849033 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:34.849078 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:34.849299 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:34.849300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:34.849328 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:34.849411 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.849414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.850779 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.850780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.851869 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.851871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.852678 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.852679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.852734 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.852737 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:34.852791 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 22:39:34.852824 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":32,"thread_start_us":89,"threads_started":1}
I20260807 22:39:34.853129 25077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.848659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.28.82.193:42111)
0807 22:39:34.848959 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.848963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.848974 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:39:34.849108 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.849111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.849130 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.849204 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:34.849521 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.850676 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.851839 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.852589 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.852678 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.852732 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.852982 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.853030 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":101,"threads_started":1}
I20260807 22:39:34.853219 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.848658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.28.82.194:42859)
0807 22:39:34.848874 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.848878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.849063 (+ 185us) server_negotiation.cc:407] Connection header received
0807 22:39:34.849111 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.849114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.849130 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.849202 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:34.849527 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.850215 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.851917 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.852425 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.852527 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.852593 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.853073 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.853126 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":96,"threads_started":1}
I20260807 22:39:34.853251 25078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.848536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42111 (local address 127.0.0.1:37304)
0807 22:39:34.848861 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.848903 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:34.848972 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:34.849296 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:34.849297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:34.849326 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:34.849438 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.849440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.850807 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.850808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.851754 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.851757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.852771 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.852773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.852838 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.852842 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:34.852890 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 22:39:34.852932 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":31,"thread_start_us":49,"threads_started":1}
I20260807 22:39:34.854017 24520 tablet_service.cc:1467] Processing CreateTablet for tablet b776a9f78dba4af0a7ad67cd1bd7a7da (DEFAULT_TABLE table=test-table [id=4ce2d7cd8a244d828f2d1d9e23dcf661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:34.854017 24724 tablet_service.cc:1467] Processing CreateTablet for tablet b776a9f78dba4af0a7ad67cd1bd7a7da (DEFAULT_TABLE table=test-table [id=4ce2d7cd8a244d828f2d1d9e23dcf661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:34.854379 24520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776a9f78dba4af0a7ad67cd1bd7a7da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:34.854379 24724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776a9f78dba4af0a7ad67cd1bd7a7da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:34.854389 24957 tablet_service.cc:1467] Processing CreateTablet for tablet b776a9f78dba4af0a7ad67cd1bd7a7da (DEFAULT_TABLE table=test-table [id=4ce2d7cd8a244d828f2d1d9e23dcf661]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:34.854784 24957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776a9f78dba4af0a7ad67cd1bd7a7da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:34.857441 25082 tablet_bootstrap.cc:492] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: Bootstrap starting.
I20260807 22:39:34.857858 25084 tablet_bootstrap.cc:492] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04: Bootstrap starting.
I20260807 22:39:34.857859 25083 tablet_bootstrap.cc:492] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Bootstrap starting.
I20260807 22:39:34.858270 25082 tablet_bootstrap.cc:654] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:34.858572 25083 tablet_bootstrap.cc:654] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:34.858572 25084 tablet_bootstrap.cc:654] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:34.858891 25082 log.cc:824] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:34.859195 25084 log.cc:824] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:34.859207 25083 log.cc:824] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:34.859750 25082 tablet_bootstrap.cc:492] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: No bootstrap required, opened a new log
I20260807 22:39:34.859876 25082 ts_tablet_manager.cc:1397] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.002s
I20260807 22:39:34.859987 25083 tablet_bootstrap.cc:492] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: No bootstrap required, opened a new log
I20260807 22:39:34.860065 25083 ts_tablet_manager.cc:1397] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:39:34.860239 25084 tablet_bootstrap.cc:492] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04: No bootstrap required, opened a new log
I20260807 22:39:34.860308 25084 ts_tablet_manager.cc:1397] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:39:34.861457 25082 raft_consensus.cc:348] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.861631 25082 raft_consensus.cc:374] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:34.861680 25082 raft_consensus.cc:729] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f632c33740d408ca6941837b5dcf5e1, State: Initialized, Role: FOLLOWER
I20260807 22:39:34.861793 25082 consensus_queue.cc:260] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.862069 25082 ts_tablet_manager.cc:1428] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 22:39:34.862219 25055 heartbeater.cc:503] Master 127.28.82.254:45499 was elected leader, sending a full tablet report...
I20260807 22:39:34.862152 25083 raft_consensus.cc:348] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.862214 25084 raft_consensus.cc:348] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.862358 25084 raft_consensus.cc:374] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:34.862354 25083 raft_consensus.cc:374] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:34.862401 25083 raft_consensus.cc:729] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dbc5691456c493e82e8744c9733dc7b, State: Initialized, Role: FOLLOWER
I20260807 22:39:34.862401 25084 raft_consensus.cc:729] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b5b38129e446d689be6f2f2315ba04, State: Initialized, Role: FOLLOWER
I20260807 22:39:34.862540 25084 consensus_queue.cc:260] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.862596 25083 consensus_queue.cc:260] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.862824 25084 ts_tablet_manager.cc:1428] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:39:34.862864 25083 ts_tablet_manager.cc:1428] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 22:39:34.862960 24622 heartbeater.cc:503] Master 127.28.82.254:45499 was elected leader, sending a full tablet report...
I20260807 22:39:34.863332 24836 heartbeater.cc:503] Master 127.28.82.254:45499 was elected leader, sending a full tablet report...
I20260807 22:39:34.884912 25092 raft_consensus.cc:482] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:34.885020 25092 raft_consensus.cc:504] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.885324 25092 leader_election.cc:288] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dbc5691456c493e82e8744c9733dc7b (127.28.82.194:42859), 2f632c33740d408ca6941837b5dcf5e1 (127.28.82.195:46041)
I20260807 22:39:34.888444 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.885503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:42859 (local address 127.28.82.193:38111)
0807 22:39:34.885573 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.885586 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:34.885628 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:34.885794 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:34.885796 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:34.885798 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:34.885881 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.885882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.886687 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.886688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.887516 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.887518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.888147 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.888148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.888250 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.888255 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:39:34.888311 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 22:39:34.888356 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 22:39:34.888463 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.885599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:38111 (local address 127.28.82.194:42859)
0807 22:39:34.885657 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.885659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.885667 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:39:34.885693 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.885694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.885703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.885767 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:34.885957 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.886609 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.887610 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.888042 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.888099 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.888106 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.888330 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.888371 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:39:34.888629 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.885585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:45973 (local address 127.28.82.195:46041)
0807 22:39:34.885649 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.885652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.885966 (+ 314us) server_negotiation.cc:407] Connection header received
0807 22:39:34.886017 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.886018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.886026 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.886062 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:34.886198 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.886802 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.887766 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.888201 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.888283 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.888293 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.888475 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.888541 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:39:34.888772 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.885503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:46041 (local address 127.28.82.193:45973)
0807 22:39:34.885873 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.885883 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:34.885958 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:34.886066 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:34.886066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:34.886067 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:34.886137 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.886139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.886885 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.886886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.887659 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:34.887661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.888312 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.888313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:34.888385 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.888389 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:34.888448 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:39:34.888489 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":154,"thread_start_us":136,"threads_started":1}
I20260807 22:39:34.888797 24756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776a9f78dba4af0a7ad67cd1bd7a7da" candidate_uuid: "51b5b38129e446d689be6f2f2315ba04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dbc5691456c493e82e8744c9733dc7b" is_pre_election: true
I20260807 22:39:34.888994 24756 raft_consensus.cc:2454] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b5b38129e446d689be6f2f2315ba04 in term 0.
I20260807 22:39:34.889127 24987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776a9f78dba4af0a7ad67cd1bd7a7da" candidate_uuid: "51b5b38129e446d689be6f2f2315ba04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f632c33740d408ca6941837b5dcf5e1" is_pre_election: true
I20260807 22:39:34.889256 24450 leader_election.cc:302] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dbc5691456c493e82e8744c9733dc7b, 51b5b38129e446d689be6f2f2315ba04; no voters:
I20260807 22:39:34.889307 24987 raft_consensus.cc:2454] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b5b38129e446d689be6f2f2315ba04 in term 0.
I20260807 22:39:34.889400 25092 raft_consensus.cc:2781] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:39:34.889464 25092 raft_consensus.cc:482] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:39:34.889497 25092 raft_consensus.cc:3037] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:34.890249 25092 raft_consensus.cc:504] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.890399 25092 leader_election.cc:288] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [CANDIDATE]: Term 1 election: Requested vote from peers 0dbc5691456c493e82e8744c9733dc7b (127.28.82.194:42859), 2f632c33740d408ca6941837b5dcf5e1 (127.28.82.195:46041)
I20260807 22:39:34.890607 24987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776a9f78dba4af0a7ad67cd1bd7a7da" candidate_uuid: "51b5b38129e446d689be6f2f2315ba04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f632c33740d408ca6941837b5dcf5e1"
I20260807 22:39:34.890599 24756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776a9f78dba4af0a7ad67cd1bd7a7da" candidate_uuid: "51b5b38129e446d689be6f2f2315ba04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dbc5691456c493e82e8744c9733dc7b"
I20260807 22:39:34.890687 24756 raft_consensus.cc:3037] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:34.890687 24987 raft_consensus.cc:3037] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:34.891584 24756 raft_consensus.cc:2454] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b5b38129e446d689be6f2f2315ba04 in term 1.
I20260807 22:39:34.891584 24987 raft_consensus.cc:2454] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b5b38129e446d689be6f2f2315ba04 in term 1.
I20260807 22:39:34.891753 24450 leader_election.cc:302] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dbc5691456c493e82e8744c9733dc7b, 51b5b38129e446d689be6f2f2315ba04; no voters:
I20260807 22:39:34.891855 25092 raft_consensus.cc:2781] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:34.892014 25092 raft_consensus.cc:686] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [term 1 LEADER]: Becoming Leader. State: Replica: 51b5b38129e446d689be6f2f2315ba04, State: Running, Role: LEADER
I20260807 22:39:34.892117 25092 consensus_queue.cc:237] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } }
I20260807 22:39:34.892930 24335 catalog_manager.cc:5495] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b5b38129e446d689be6f2f2315ba04 (127.28.82.193). New cstate: current_term: 1 leader_uuid: "51b5b38129e446d689be6f2f2315ba04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51b5b38129e446d689be6f2f2315ba04" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 42111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 } health_report { overall_health: UNKNOWN } } }
W20260807 22:39:34.902551 24839 tablet.cc:2369] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:39:34.909801 25077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.907627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37312 (local address 127.28.82.193:42111)
0807 22:39:34.907721 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:34.907723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:34.907738 (+ 15us) server_negotiation.cc:407] Connection header received
0807 22:39:34.907767 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:34.907768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:34.907783 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:34.907852 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:39:34.907985 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.908441 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.909122 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:34.909307 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:34.909355 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:34.909492 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:39:34.909581 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:39:34.909642 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:34.909680 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 22:39:34.909697 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 22:39:34.911587 24756 raft_consensus.cc:1261] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 1 FOLLOWER]: Refusing update from remote peer 51b5b38129e446d689be6f2f2315ba04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:34.911592 24987 raft_consensus.cc:1261] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 1 FOLLOWER]: Refusing update from remote peer 51b5b38129e446d689be6f2f2315ba04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:34.911958 25092 consensus_queue.cc:1035] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dbc5691456c493e82e8744c9733dc7b" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 42859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:34.912107 25098 consensus_queue.cc:1035] T b776a9f78dba4af0a7ad67cd1bd7a7da P 51b5b38129e446d689be6f2f2315ba04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f632c33740d408ca6941837b5dcf5e1" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:34.914299 25101 mvcc.cc:205] Tried to move back new op lower bound from 7316039167635263488 to 7316039167558782976. Current Snapshot: MvccSnapshot[applied={T|T < 7316039167635263488}]
I20260807 22:39:34.914392 25104 mvcc.cc:205] Tried to move back new op lower bound from 7316039167635263488 to 7316039167558782976. Current Snapshot: MvccSnapshot[applied={T|T < 7316039167635263488}]
I20260807 22:39:34.916803 24332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:34.916910 24332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:34.917676 24332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7d3a64fae304c1698194e8dca9b9014: Sending DeleteTablet for 3 replicas of tablet b776a9f78dba4af0a7ad67cd1bd7a7da
I20260807 22:39:34.917987 24520 tablet_service.cc:1514] Processing DeleteTablet for tablet b776a9f78dba4af0a7ad67cd1bd7a7da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:34 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:37304
I20260807 22:39:34.918037 24957 tablet_service.cc:1514] Processing DeleteTablet for tablet b776a9f78dba4af0a7ad67cd1bd7a7da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:34 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:60276
I20260807 22:39:34.918108 24724 tablet_service.cc:1514] Processing DeleteTablet for tablet b776a9f78dba4af0a7ad67cd1bd7a7da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:34 UTC) from {username='kudu', principal='kudu/127.28.82.254@KRBTEST.COM'} at 127.0.0.1:34698
I20260807 22:39:34.918361 25119 tablet_replica.cc:330] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: stopping tablet replica
I20260807 22:39:34.918434 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 24420
I20260807 22:39:34.918493 25119 raft_consensus.cc:2229] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:34.918591 25119 raft_consensus.cc:2258] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:34.923453 25119 ts_tablet_manager.cc:1905] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:34.924554 25118 tablet_replica.cc:330] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: stopping tablet replica
I20260807 22:39:34.924654 25118 raft_consensus.cc:2229] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:34.924739 25118 raft_consensus.cc:2258] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:34.925736 25119 ts_tablet_manager.cc:1918] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:34.925835 25119 log.cc:1196] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/b776a9f78dba4af0a7ad67cd1bd7a7da
I20260807 22:39:34.926198 25119 ts_tablet_manager.cc:1939] T b776a9f78dba4af0a7ad67cd1bd7a7da P 0dbc5691456c493e82e8744c9733dc7b: Deleting consensus metadata
I20260807 22:39:34.926699 24309 catalog_manager.cc:4841] TS 0dbc5691456c493e82e8744c9733dc7b (127.28.82.194:42859): tablet b776a9f78dba4af0a7ad67cd1bd7a7da (table test-table [id=4ce2d7cd8a244d828f2d1d9e23dcf661]) successfully deleted
I20260807 22:39:34.926880 25118 ts_tablet_manager.cc:1905] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:34.928072 25118 ts_tablet_manager.cc:1918] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:34.928138 25118 log.cc:1196] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/b776a9f78dba4af0a7ad67cd1bd7a7da
I20260807 22:39:34.928418 25118 ts_tablet_manager.cc:1939] T b776a9f78dba4af0a7ad67cd1bd7a7da P 2f632c33740d408ca6941837b5dcf5e1: Deleting consensus metadata
I20260807 22:39:34.928860 24312 catalog_manager.cc:4841] TS 2f632c33740d408ca6941837b5dcf5e1 (127.28.82.195:46041): tablet b776a9f78dba4af0a7ad67cd1bd7a7da (table test-table [id=4ce2d7cd8a244d828f2d1d9e23dcf661]) successfully deleted
W20260807 22:39:34.932185 24312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:42111 (error 108)
I20260807 22:39:34.932461 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 24627
I20260807 22:39:34.938596 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:34.938390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:42111 (local address 127.0.0.1:37322)
0807 22:39:34.938458 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:34.938546 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:42111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 22:39:34.938766 24312 catalog_manager.cc:4584] TS 51b5b38129e446d689be6f2f2315ba04 (127.28.82.193:42111): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b776a9f78dba4af0a7ad67cd1bd7a7da: Network error: Client connection negotiation failed: client connection to 127.28.82.193:42111: connect: Connection refused (error 111)
I20260807 22:39:34.943104 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 24845
I20260807 22:39:34.955690 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 24287
2026-08-07T22:39:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2131 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T22:39:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:34Z Disabled control of system clock
I20260807 22:39:34.989696 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37301
--webserver_interface=127.28.82.254
--webserver_port=0
--builtin_ntp_servers=127.28.82.212:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:37301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:35.072175 25127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:35.072381 25127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:35.072402 25127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:35.073474 25127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:35.073514 25127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:35.073527 25127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:35.073545 25127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:35.073559 25127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:35.074903 25127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:37301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:37301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:35.075088 25127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:35.075305 25127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:35.080260 25133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:35.080349 25127 server_base.cc:1034] running on GCE node
W20260807 22:39:35.080293 25134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:35.080271 25139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:35.080758 25127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:35.081099 25127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:35.082304 25127 hybrid_clock.cc:648] HybridClock initialized: now 1786142375082286 us; error 52 us; skew 500 ppm
I20260807 22:39:35.083624 25127 webserver.cc:460] Webserver started at http://127.28.82.254:42227/ using document root <none> and password file <none>
I20260807 22:39:35.083871 25127 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:35.083920 25127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:35.084070 25127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:35.085131 25127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "6c1f0f29af744159bbcd43a71f9d9d7c"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "37d1d7f3db58a4d85fcdcc470b21b407"
server_key_iv: "71667cde464c6b44b8b7019ec5840462"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.085500 25127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "6c1f0f29af744159bbcd43a71f9d9d7c"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "37d1d7f3db58a4d85fcdcc470b21b407"
server_key_iv: "71667cde464c6b44b8b7019ec5840462"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.086993 25127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:35.087872 25151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.088109 25127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:35.088207 25127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "6c1f0f29af744159bbcd43a71f9d9d7c"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "37d1d7f3db58a4d85fcdcc470b21b407"
server_key_iv: "71667cde464c6b44b8b7019ec5840462"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.088285 25127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:35.102082 25127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:35.102371 25127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:35.102600 25127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:35.106590 25127 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:37301
I20260807 22:39:35.106675 25232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:37301 every 8 connection(s)
I20260807 22:39:35.106999 25127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:35.107995 25233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:35.110193 25233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c: Bootstrap starting.
I20260807 22:39:35.110817 25233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:35.111088 25233 log.cc:824] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:35.111737 25233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c: No bootstrap required, opened a new log
I20260807 22:39:35.113296 25233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1f0f29af744159bbcd43a71f9d9d7c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37301 } }
I20260807 22:39:35.113432 25233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:35.113456 25233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1f0f29af744159bbcd43a71f9d9d7c, State: Initialized, Role: FOLLOWER
I20260807 22:39:35.113590 25233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1f0f29af744159bbcd43a71f9d9d7c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37301 } }
I20260807 22:39:35.113672 25233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:35.113694 25233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:35.113727 25233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:35.114243 25233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1f0f29af744159bbcd43a71f9d9d7c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37301 } }
I20260807 22:39:35.114364 25233 leader_election.cc:302] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c1f0f29af744159bbcd43a71f9d9d7c; no voters:
I20260807 22:39:35.114391 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 25127
I20260807 22:39:35.114549 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:35.114586 25233 leader_election.cc:288] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:35.114705 25236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:35.114797 29003 external_mini_cluster.cc:1428] Setting key 1dfbfdd9f1728ef275e7e66d210b9e2d
I20260807 22:39:35.114902 25236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [term 1 LEADER]: Becoming Leader. State: Replica: 6c1f0f29af744159bbcd43a71f9d9d7c, State: Running, Role: LEADER
I20260807 22:39:35.114913 25233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:35.115032 25236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1f0f29af744159bbcd43a71f9d9d7c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37301 } }
I20260807 22:39:35.115342 25236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c1f0f29af744159bbcd43a71f9d9d7c. Latest consensus state: current_term: 1 leader_uuid: "6c1f0f29af744159bbcd43a71f9d9d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1f0f29af744159bbcd43a71f9d9d7c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37301 } } }
I20260807 22:39:35.115408 25236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:35.115583 25239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c1f0f29af744159bbcd43a71f9d9d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1f0f29af744159bbcd43a71f9d9d7c" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 37301 } } }
I20260807 22:39:35.115682 25239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:35.115669 25244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:35.116489 25244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:35.118274 25244 catalog_manager.cc:1347] Generated new cluster ID: 717c37d61c5546e59d0232190a82c415
I20260807 22:39:35.118336 25244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:35.121235 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.115785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40488 (local address 127.28.82.254:37301)
0807 22:39:35.116202 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.116211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.116256 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:39:35.116376 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.116384 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.116469 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.116891 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:35.116999 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.118175 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.119138 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.119569 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.119704 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.120476 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:35.120480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:35.120480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:35.120660 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:35.120698 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:35.120708 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:35.120789 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.120893 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.121077 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":61,"threads_started":1}
I20260807 22:39:35.127033 25244 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:35.127734 25244 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:35.135360 25244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c: Generated new TSK 0
I20260807 22:39:35.135532 25244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:35.140676 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.193:0
--local_ip_for_outbound_sockets=127.28.82.193
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37301
--builtin_ntp_servers=127.28.82.212:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 22:39:35.236038 25270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:35.236353 25270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:35.236397 25270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:35.238507 25270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:35.238574 25270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:35.238621 25270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.193
I20260807 22:39:35.241091 25270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.82.193
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.193
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:35.241379 25270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:35.241782 25270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:35.247565 25281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:35.247802 25285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:35.247906 25270 server_base.cc:1034] running on GCE node
W20260807 22:39:35.247843 25280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:35.248208 25270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:35.248517 25270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:35.249707 25270 hybrid_clock.cc:648] HybridClock initialized: now 1786142375249694 us; error 29 us; skew 500 ppm
I20260807 22:39:35.251391 25270 webserver.cc:460] Webserver started at http://127.28.82.193:38397/ using document root <none> and password file <none>
I20260807 22:39:35.251716 25270 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:35.251804 25270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:35.251961 25270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:35.253511 25270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/data/instance:
uuid: "351c5d8d9cce4504ad46d3af47bd4433"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "48e73f25f7823722b5639d9b68a91ce4"
server_key_iv: "445621255efc4635c6856fd82a443908"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.253975 25270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance:
uuid: "351c5d8d9cce4504ad46d3af47bd4433"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "48e73f25f7823722b5639d9b68a91ce4"
server_key_iv: "445621255efc4635c6856fd82a443908"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.255877 25270 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 22:39:35.256866 25293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.257058 25270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:35.257143 25270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/wal
uuid: "351c5d8d9cce4504ad46d3af47bd4433"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "48e73f25f7823722b5639d9b68a91ce4"
server_key_iv: "445621255efc4635c6856fd82a443908"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.257215 25270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:35.274588 25270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:35.274915 25270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:35.275072 25270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:35.275324 25270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:35.275668 25270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:35.275725 25270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.275772 25270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:35.275804 25270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.282539 25270 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.193:40169
I20260807 22:39:35.282585 25464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.193:40169 every 8 connection(s)
I20260807 22:39:35.282929 25270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/data/info.pb
I20260807 22:39:35.287266 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 25270
I20260807 22:39:35.287393 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-0/wal/instance
I20260807 22:39:35.287624 29003 external_mini_cluster.cc:1428] Setting key 62cd150fdda81d089f49b7b1428336ce
I20260807 22:39:35.288767 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.194:0
--local_ip_for_outbound_sockets=127.28.82.194
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37301
--builtin_ntp_servers=127.28.82.212:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:39:35.289238 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.283724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.193:49767 (local address 127.28.82.254:37301)
0807 22:39:35.283797 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.283800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.284045 (+ 245us) server_negotiation.cc:407] Connection header received
0807 22:39:35.284591 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.284593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.284611 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.284675 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:35.285428 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.286271 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.287400 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.287640 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.287704 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.288836 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:35.288838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:35.288838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:35.288844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:35.288864 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:35.288870 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:35.288930 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.289081 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.289117 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:39:35.289460 25468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.283699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37301 (local address 127.28.82.193:49767)
0807 22:39:35.283908 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.283953 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:35.284480 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:35.284793 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:35.284797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:35.284806 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:35.285327 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.285331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.286372 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.286372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.287249 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.287252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.287822 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.287823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.287984 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.288664 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:35.288664 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:35.288705 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:35.288735 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:35.288961 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:35.288969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:35.288972 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:35.289069 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 22:39:35.289186 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260807 22:39:35.289933 25465 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37301
I20260807 22:39:35.290031 25465 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:35.290239 25465 heartbeater.cc:511] Master 127.28.82.254:37301 requested a full tablet report, sending...
I20260807 22:39:35.290812 25180 ts_manager.cc:194] Registered new tserver with Master: 351c5d8d9cce4504ad46d3af47bd4433 (127.28.82.193:40169)
I20260807 22:39:35.291448 25180 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.193:49767
W20260807 22:39:35.372956 25469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:35.373214 25469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:35.373252 25469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:35.374414 25469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:35.374511 25469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:35.374564 25469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.194
I20260807 22:39:35.375878 25469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.82.194
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.194
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:35.376060 25469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:35.376305 25469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:35.381536 25481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:35.381553 25478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:35.381538 25475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:35.381740 25469 server_base.cc:1034] running on GCE node
I20260807 22:39:35.382030 25469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:35.382304 25469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:35.383474 25469 hybrid_clock.cc:648] HybridClock initialized: now 1786142375383460 us; error 48 us; skew 500 ppm
I20260807 22:39:35.384709 25469 webserver.cc:460] Webserver started at http://127.28.82.194:39609/ using document root <none> and password file <none>
I20260807 22:39:35.384950 25469 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:35.385021 25469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:35.385159 25469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:35.386237 25469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/data/instance:
uuid: "6222dc65f238415894ca42c305161c4d"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "14921a71851a8634ef298e33d34c08ba"
server_key_iv: "baeb97078d9560cab3e18a26e5ee6ec5"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.386638 25469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance:
uuid: "6222dc65f238415894ca42c305161c4d"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "14921a71851a8634ef298e33d34c08ba"
server_key_iv: "baeb97078d9560cab3e18a26e5ee6ec5"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.387982 25469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:35.388732 25491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.388921 25469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:39:35.389024 25469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/wal
uuid: "6222dc65f238415894ca42c305161c4d"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "14921a71851a8634ef298e33d34c08ba"
server_key_iv: "baeb97078d9560cab3e18a26e5ee6ec5"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.389112 25469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:35.397322 25469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:35.397589 25469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:35.397740 25469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:35.397962 25469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:35.398283 25469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:35.398341 25469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.398380 25469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:35.398408 25469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.404517 25469 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.194:37733
I20260807 22:39:35.404582 25658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.194:37733 every 8 connection(s)
I20260807 22:39:35.404878 25469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/data/info.pb
I20260807 22:39:35.405169 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 25469
I20260807 22:39:35.405287 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/wal/instance
I20260807 22:39:35.405540 29003 external_mini_cluster.cc:1428] Setting key 3eb8305baf30ac1ec503a419f9662290
I20260807 22:39:35.406575 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.82.195:0
--local_ip_for_outbound_sockets=127.28.82.195
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37301
--builtin_ntp_servers=127.28.82.212:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 22:39:35.410923 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.405622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.194:51559 (local address 127.28.82.254:37301)
0807 22:39:35.405677 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.405679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.406054 (+ 375us) server_negotiation.cc:407] Connection header received
0807 22:39:35.406857 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.406858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.406879 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.406936 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:35.407840 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.408336 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.409340 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.409567 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.409686 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.410602 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:35.410603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:35.410604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:35.410609 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:35.410625 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:35.410630 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:35.410703 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.410804 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.410840 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 22:39:35.411166 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.405661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37301 (local address 127.28.82.194:51559)
0807 22:39:35.405913 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.405954 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:35.406714 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:35.407005 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:35.407010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:35.407020 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:35.407738 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.407745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.408452 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.408454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.409240 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.409242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.409662 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.409664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.409810 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.410403 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:35.410403 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:35.410432 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:35.410455 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:35.410705 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:35.410713 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:35.410717 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:35.410807 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 22:39:35.410915 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":101,"threads_started":1}
I20260807 22:39:35.411635 25659 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37301
I20260807 22:39:35.411729 25659 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:35.411931 25659 heartbeater.cc:511] Master 127.28.82.254:37301 requested a full tablet report, sending...
I20260807 22:39:35.412322 25180 ts_manager.cc:194] Registered new tserver with Master: 6222dc65f238415894ca42c305161c4d (127.28.82.194:37733)
I20260807 22:39:35.412705 25180 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.194:51559
W20260807 22:39:35.490101 25662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:35.490334 25662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:35.490366 25662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:35.491601 25662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:35.491659 25662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:35.491703 25662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.82.195
I20260807 22:39:35.493131 25662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.82.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.82.195
--webserver_port=0
--tserver_master_addrs=127.28.82.254:37301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.82.195
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:35.493392 25662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:35.493696 25662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:35.498859 25668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:35.498960 25671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:35.499042 25662 server_base.cc:1034] running on GCE node
W20260807 22:39:35.498854 25669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:35.499305 25662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:35.499553 25662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:35.500766 25662 hybrid_clock.cc:648] HybridClock initialized: now 1786142375500742 us; error 28 us; skew 500 ppm
I20260807 22:39:35.502130 25662 webserver.cc:460] Webserver started at http://127.28.82.195:38641/ using document root <none> and password file <none>
I20260807 22:39:35.502403 25662 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:35.502497 25662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:35.502679 25662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:35.503763 25662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/data/instance:
uuid: "2ea0ce9134d840cd85a0b3525753e21f"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "a44d19e2e29f00a50d502b65862d4df1"
server_key_iv: "074c735e45ae109e4347e5aa43162d08"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.504137 25662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance:
uuid: "2ea0ce9134d840cd85a0b3525753e21f"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "a44d19e2e29f00a50d502b65862d4df1"
server_key_iv: "074c735e45ae109e4347e5aa43162d08"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.505466 25662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:35.506251 25678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.506425 25662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:39:35.506559 25662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/wal
uuid: "2ea0ce9134d840cd85a0b3525753e21f"
format_stamp: "Formatted at 2026-08-07 22:39:35 on dist-test-slave-7kzw"
server_key: "a44d19e2e29f00a50d502b65862d4df1"
server_key_iv: "074c735e45ae109e4347e5aa43162d08"
server_key_version: "encryptionkey@0"
I20260807 22:39:35.506639 25662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:35.513079 25662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:35.513350 25662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:35.513495 25662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:35.513718 25662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 22:39:35.514034 25662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 22:39:35.514091 25662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.514130 25662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 22:39:35.514158 25662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:35.520311 25662 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.195:46459
I20260807 22:39:35.520377 25844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.195:46459 every 8 connection(s)
I20260807 22:39:35.520706 25662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/data/info.pb
I20260807 22:39:35.522946 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 25662
I20260807 22:39:35.523082 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/wal/instance
I20260807 22:39:35.523280 29003 external_mini_cluster.cc:1428] Setting key 8e6733c8c8b52a8f277a014fac0767db
I20260807 22:39:35.526316 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.521457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:49691 (local address 127.28.82.254:37301)
0807 22:39:35.521521 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.521523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.521784 (+ 261us) server_negotiation.cc:407] Connection header received
0807 22:39:35.522302 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.522304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.522321 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.522386 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:35.523207 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.523899 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.524930 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.525148 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.525228 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.526007 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:35.526009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:35.526009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:35.526014 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:35.526030 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:35.526034 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:35.526096 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.526205 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.526236 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:39:35.526604 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.521448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:37301 (local address 127.28.82.195:49691)
0807 22:39:35.521668 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.521699 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:35.522195 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:35.522500 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:35.522505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:35.522526 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:39:35.523055 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.523060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.523987 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.523988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.524831 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.524834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.525238 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.525239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.525370 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.525869 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 22:39:35.525869 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:39:35.525895 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 22:39:35.525916 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:39:35.526100 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:39:35.526105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:39:35.526108 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 22:39:35.526187 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 22:39:35.526288 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260807 22:39:35.527103 25846 heartbeater.cc:347] Connected to a master server at 127.28.82.254:37301
I20260807 22:39:35.527181 25846 heartbeater.cc:464] Registering TS with master...
I20260807 22:39:35.527393 25846 heartbeater.cc:511] Master 127.28.82.254:37301 requested a full tablet report, sending...
I20260807 22:39:35.527820 25180 ts_manager.cc:194] Registered new tserver with Master: 2ea0ce9134d840cd85a0b3525753e21f (127.28.82.195:46459)
I20260807 22:39:35.528157 25180 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.82.195:49691
I20260807 22:39:35.535156 29003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 22:39:35.539531 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.536581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40504 (local address 127.28.82.254:37301)
0807 22:39:35.536692 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.536695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.536740 (+ 45us) server_negotiation.cc:407] Connection header received
0807 22:39:35.536816 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.536817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.536828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.536878 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:35.537054 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.537515 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.538274 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.538502 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.538581 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.539295 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:35.539296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:35.539296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:35.539301 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:35.539316 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:35.539320 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:35.539368 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.539424 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.539447 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 22:39:35.541218 25180 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 22:39:35.542786 25180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 22:39:35.549208 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.545045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.28.82.195:46459)
0807 22:39:35.545464 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.545469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.545481 (+ 12us) server_negotiation.cc:407] Connection header received
0807 22:39:35.545611 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.545614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.545630 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.545709 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:35.545902 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.547076 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.548306 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.548792 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.548892 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.548983 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.549084 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.549117 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":81,"threads_started":1}
I20260807 22:39:35.549299 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.544929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:40169 (local address 127.0.0.1:44704)
0807 22:39:35.545104 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.545144 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:35.545208 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:35.545661 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:35.545665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:35.545668 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:35.545752 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.545754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.546848 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.546849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.548081 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.548084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.549017 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.549018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.549097 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.549103 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 22:39:35.549135 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 22:39:35.549177 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260807 22:39:35.549193 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.544931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.195:46459 (local address 127.0.0.1:44708)
0807 22:39:35.545425 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.545437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:35.545494 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:35.545772 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:35.545773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:35.545775 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:35.545824 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.545827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.547164 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.547164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.548216 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.548218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.548898 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.548899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.548981 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.548995 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 22:39:35.549069 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:39:35.549120 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":55,"threads_started":1}
I20260807 22:39:35.549702 25860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.545226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.28.82.193:40169)
0807 22:39:35.545440 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.545446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.545457 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:39:35.545497 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.545499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.545513 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.545837 (+ 324us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:35.545902 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.546763 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.548174 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.548909 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.549240 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.549529 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.549564 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.549611 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260807 22:39:35.549768 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.545144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:37733 (local address 127.0.0.1:46854)
0807 22:39:35.545638 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.545647 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:35.545693 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:35.546000 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:35.546001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:35.546003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:35.546044 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.546045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.547320 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.547321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.548266 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.548269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.549496 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.549497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.549564 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.549569 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 22:39:35.549650 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 22:39:35.549685 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":81,"threads_started":1}
I20260807 22:39:35.550078 25751 tablet_service.cc:1467] Processing CreateTablet for tablet 630d03b67e234e22b904e968c6abfa16 (DEFAULT_TABLE table=test-table [id=849d791fe23f423f8eb80ea77e1904bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:35.550390 25372 tablet_service.cc:1467] Processing CreateTablet for tablet 630d03b67e234e22b904e968c6abfa16 (DEFAULT_TABLE table=test-table [id=849d791fe23f423f8eb80ea77e1904bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:35.550550 25751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630d03b67e234e22b904e968c6abfa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:35.550746 25372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630d03b67e234e22b904e968c6abfa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:35.550771 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.545240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46854 (local address 127.28.82.194:37733)
0807 22:39:35.545445 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.545448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.545779 (+ 331us) server_negotiation.cc:407] Connection header received
0807 22:39:35.545891 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.545893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.545909 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.545988 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:35.546131 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.547224 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.548365 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.549092 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.549156 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.549245 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.550543 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.550687 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260807 22:39:35.551663 25558 tablet_service.cc:1467] Processing CreateTablet for tablet 630d03b67e234e22b904e968c6abfa16 (DEFAULT_TABLE table=test-table [id=849d791fe23f423f8eb80ea77e1904bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 22:39:35.552006 25558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630d03b67e234e22b904e968c6abfa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:35.553649 25862 tablet_bootstrap.cc:492] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433: Bootstrap starting.
I20260807 22:39:35.553911 25863 tablet_bootstrap.cc:492] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Bootstrap starting.
I20260807 22:39:35.554402 25862 tablet_bootstrap.cc:654] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:35.554597 25863 tablet_bootstrap.cc:654] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:35.554826 25862 log.cc:824] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:35.554922 25863 log.cc:824] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:35.555471 25866 tablet_bootstrap.cc:492] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Bootstrap starting.
I20260807 22:39:35.555652 25863 tablet_bootstrap.cc:492] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: No bootstrap required, opened a new log
I20260807 22:39:35.555650 25862 tablet_bootstrap.cc:492] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433: No bootstrap required, opened a new log
I20260807 22:39:35.555729 25863 ts_tablet_manager.cc:1397] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 22:39:35.555742 25862 ts_tablet_manager.cc:1397] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 22:39:35.556133 25866 tablet_bootstrap.cc:654] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:35.556612 25866 log.cc:824] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:35.557439 25866 tablet_bootstrap.cc:492] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: No bootstrap required, opened a new log
I20260807 22:39:35.557510 25866 ts_tablet_manager.cc:1397] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 22:39:35.558065 25862 raft_consensus.cc:348] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.558229 25862 raft_consensus.cc:374] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:35.558275 25862 raft_consensus.cc:729] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351c5d8d9cce4504ad46d3af47bd4433, State: Initialized, Role: FOLLOWER
I20260807 22:39:35.558388 25862 consensus_queue.cc:260] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.558511 25863 raft_consensus.cc:348] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.558673 25863 raft_consensus.cc:374] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:35.558696 25862 ts_tablet_manager.cc:1428] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 22:39:35.558719 25863 raft_consensus.cc:729] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea0ce9134d840cd85a0b3525753e21f, State: Initialized, Role: FOLLOWER
I20260807 22:39:35.558817 25465 heartbeater.cc:503] Master 127.28.82.254:37301 was elected leader, sending a full tablet report...
I20260807 22:39:35.558866 25863 consensus_queue.cc:260] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.559257 25863 ts_tablet_manager.cc:1428] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 22:39:35.559332 25846 heartbeater.cc:503] Master 127.28.82.254:37301 was elected leader, sending a full tablet report...
I20260807 22:39:35.560456 25866 raft_consensus.cc:348] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.560581 25866 raft_consensus.cc:374] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:35.560604 25866 raft_consensus.cc:729] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6222dc65f238415894ca42c305161c4d, State: Initialized, Role: FOLLOWER
I20260807 22:39:35.560755 25866 consensus_queue.cc:260] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.561076 25866 ts_tablet_manager.cc:1428] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 22:39:35.561141 25659 heartbeater.cc:503] Master 127.28.82.254:37301 was elected leader, sending a full tablet report...
W20260807 22:39:35.655444 25660 tablet.cc:2369] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:39:35.736816 25869 raft_consensus.cc:482] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 22:39:35.736948 25869 raft_consensus.cc:504] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.737344 25869 leader_election.cc:288] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 351c5d8d9cce4504ad46d3af47bd4433 (127.28.82.193:40169), 6222dc65f238415894ca42c305161c4d (127.28.82.194:37733)
I20260807 22:39:35.741257 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.737768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:33053 (local address 127.28.82.194:37733)
0807 22:39:35.737836 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.737838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.738104 (+ 266us) server_negotiation.cc:407] Connection header received
0807 22:39:35.738150 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.738151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.738161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.738218 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:35.738357 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.739078 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.740332 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.740886 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.740955 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.740962 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.741140 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.741184 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 22:39:35.741258 25860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.737685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.82.195:37535 (local address 127.28.82.193:40169)
0807 22:39:35.737752 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.737754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.737765 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:39:35.737799 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.737800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.737809 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.737892 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 22:39:35.738055 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.738871 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.740270 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.740859 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.740950 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.740959 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.741141 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.741195 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:39:35.741254 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.737578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:40169 (local address 127.28.82.195:37535)
0807 22:39:35.737629 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.737644 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:35.737692 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:35.737901 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:35.737903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:35.737905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:35.737989 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.737991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.738962 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.738963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.740169 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.740171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.740962 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.740963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.741042 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.741046 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:35.741105 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 22:39:35.741157 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":3}
I20260807 22:39:35.741261 25872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.737578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.194:37733 (local address 127.28.82.195:33053)
0807 22:39:35.738030 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.738040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:39:35.738100 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:39:35.738221 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:39:35.738221 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:39:35.738223 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 22:39:35.738292 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.738294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.739152 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.739152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.740252 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:39:35.740255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.740965 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.740965 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:39:35.741064 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.741068 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 22:39:35.741124 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 22:39:35.741157 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":253,"thread_start_us":147,"threads_started":1}
I20260807 22:39:35.741660 25405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630d03b67e234e22b904e968c6abfa16" candidate_uuid: "2ea0ce9134d840cd85a0b3525753e21f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351c5d8d9cce4504ad46d3af47bd4433" is_pre_election: true
I20260807 22:39:35.741814 25405 raft_consensus.cc:2454] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea0ce9134d840cd85a0b3525753e21f in term 0.
I20260807 22:39:35.741922 25583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630d03b67e234e22b904e968c6abfa16" candidate_uuid: "2ea0ce9134d840cd85a0b3525753e21f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6222dc65f238415894ca42c305161c4d" is_pre_election: true
I20260807 22:39:35.742079 25583 raft_consensus.cc:2454] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea0ce9134d840cd85a0b3525753e21f in term 0.
I20260807 22:39:35.742048 25683 leader_election.cc:302] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea0ce9134d840cd85a0b3525753e21f, 351c5d8d9cce4504ad46d3af47bd4433; no voters:
I20260807 22:39:35.742211 25869 raft_consensus.cc:2781] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 22:39:35.742267 25869 raft_consensus.cc:482] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 22:39:35.742304 25869 raft_consensus.cc:3037] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:35.743122 25869 raft_consensus.cc:504] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.743279 25869 leader_election.cc:288] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [CANDIDATE]: Term 1 election: Requested vote from peers 351c5d8d9cce4504ad46d3af47bd4433 (127.28.82.193:40169), 6222dc65f238415894ca42c305161c4d (127.28.82.194:37733)
I20260807 22:39:35.743455 25405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630d03b67e234e22b904e968c6abfa16" candidate_uuid: "2ea0ce9134d840cd85a0b3525753e21f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351c5d8d9cce4504ad46d3af47bd4433"
I20260807 22:39:35.743498 25583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630d03b67e234e22b904e968c6abfa16" candidate_uuid: "2ea0ce9134d840cd85a0b3525753e21f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6222dc65f238415894ca42c305161c4d"
I20260807 22:39:35.743551 25405 raft_consensus.cc:3037] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:35.743567 25583 raft_consensus.cc:3037] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:35.744510 25583 raft_consensus.cc:2454] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea0ce9134d840cd85a0b3525753e21f in term 1.
I20260807 22:39:35.744510 25405 raft_consensus.cc:2454] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea0ce9134d840cd85a0b3525753e21f in term 1.
I20260807 22:39:35.744731 25679 leader_election.cc:302] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea0ce9134d840cd85a0b3525753e21f, 6222dc65f238415894ca42c305161c4d; no voters:
I20260807 22:39:35.744849 25869 raft_consensus.cc:2781] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:35.745013 25869 raft_consensus.cc:686] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 1 LEADER]: Becoming Leader. State: Replica: 2ea0ce9134d840cd85a0b3525753e21f, State: Running, Role: LEADER
I20260807 22:39:35.745111 25869 consensus_queue.cc:237] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } }
I20260807 22:39:35.745987 25180 catalog_manager.cc:5495] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ea0ce9134d840cd85a0b3525753e21f (127.28.82.195). New cstate: current_term: 1 leader_uuid: "2ea0ce9134d840cd85a0b3525753e21f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea0ce9134d840cd85a0b3525753e21f" member_type: VOTER last_known_addr { host: "127.28.82.195" port: 46459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 } health_report { overall_health: UNKNOWN } } }
W20260807 22:39:35.771314 25847 tablet.cc:2369] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 22:39:35.775324 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.772829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.28.82.195:46459)
0807 22:39:35.772898 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:35.772900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:35.772908 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:39:35.772943 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:35.772943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:35.772960 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:35.773030 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:39:35.773166 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.773620 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.774352 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:35.774621 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:35.774690 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:35.774871 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:39:35.774966 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:39:35.775030 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:35.775159 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 22:39:35.775204 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 22:39:35.777153 25583 raft_consensus.cc:1261] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 1 FOLLOWER]: Refusing update from remote peer 2ea0ce9134d840cd85a0b3525753e21f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:35.777151 25405 raft_consensus.cc:1261] T 630d03b67e234e22b904e968c6abfa16 P 351c5d8d9cce4504ad46d3af47bd4433 [term 1 FOLLOWER]: Refusing update from remote peer 2ea0ce9134d840cd85a0b3525753e21f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:39:35.777590 25873 consensus_queue.cc:1035] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [LEADER]: Connected to new peer: Peer: permanent_uuid: "351c5d8d9cce4504ad46d3af47bd4433" member_type: VOTER last_known_addr { host: "127.28.82.193" port: 40169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:35.777757 25869 consensus_queue.cc:1035] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6222dc65f238415894ca42c305161c4d" member_type: VOTER last_known_addr { host: "127.28.82.194" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:39:35.780367 25878 mvcc.cc:205] Tried to move back new op lower bound from 7316039171180376064 to 7316039171052785664. Current Snapshot: MvccSnapshot[applied={T|T < 7316039171180376064}]
I20260807 22:39:35.780534 25879 mvcc.cc:205] Tried to move back new op lower bound from 7316039171180376064 to 7316039171052785664. Current Snapshot: MvccSnapshot[applied={T|T < 7316039171180376064}]
I20260807 22:39:35.780699 25880 mvcc.cc:205] Tried to move back new op lower bound from 7316039171180376064 to 7316039171052785664. Current Snapshot: MvccSnapshot[applied={T|T < 7316039171180376064}]
I20260807 22:39:35.782747 25180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:35.782846 25180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 22:39:35.783700 25180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c1f0f29af744159bbcd43a71f9d9d7c: Sending DeleteTablet for 3 replicas of tablet 630d03b67e234e22b904e968c6abfa16
I20260807 22:39:35.784058 25558 tablet_service.cc:1514] Processing DeleteTablet for tablet 630d03b67e234e22b904e968c6abfa16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:35 UTC) from {username='slave'} at 127.0.0.1:46854
I20260807 22:39:35.784068 25372 tablet_service.cc:1514] Processing DeleteTablet for tablet 630d03b67e234e22b904e968c6abfa16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:35 UTC) from {username='slave'} at 127.0.0.1:44704
I20260807 22:39:35.784096 25751 tablet_service.cc:1514] Processing DeleteTablet for tablet 630d03b67e234e22b904e968c6abfa16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 22:39:35 UTC) from {username='slave'} at 127.0.0.1:44708
I20260807 22:39:35.784312 25895 tablet_replica.cc:330] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: stopping tablet replica
I20260807 22:39:35.784402 25895 raft_consensus.cc:2229] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 1 LEADER]: Raft consensus shutting down.
I20260807 22:39:35.784512 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 25270
I20260807 22:39:35.784528 25895 raft_consensus.cc:2258] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:35.785157 25894 tablet_replica.cc:330] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: stopping tablet replica
I20260807 22:39:35.785251 25894 raft_consensus.cc:2229] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 22:39:35.785336 25894 raft_consensus.cc:2258] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 22:39:35.789490 25895 ts_tablet_manager.cc:1905] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:35.791668 25895 ts_tablet_manager.cc:1918] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:35.791736 25895 log.cc:1196] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-2/wal/wals/630d03b67e234e22b904e968c6abfa16
I20260807 22:39:35.791957 25895 ts_tablet_manager.cc:1939] T 630d03b67e234e22b904e968c6abfa16 P 2ea0ce9134d840cd85a0b3525753e21f: Deleting consensus metadata
I20260807 22:39:35.794569 25894 ts_tablet_manager.cc:1905] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 22:39:35.795252 25152 catalog_manager.cc:4841] TS 2ea0ce9134d840cd85a0b3525753e21f (127.28.82.195:46459): tablet 630d03b67e234e22b904e968c6abfa16 (table test-table [id=849d791fe23f423f8eb80ea77e1904bd]) successfully deleted
I20260807 22:39:35.796764 25894 ts_tablet_manager.cc:1918] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 22:39:35.796936 25894 log.cc:1196] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Deleting WAL directory at /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786142120597695-29003-0/minicluster-data/ts-1/wal/wals/630d03b67e234e22b904e968c6abfa16
I20260807 22:39:35.797353 25894 ts_tablet_manager.cc:1939] T 630d03b67e234e22b904e968c6abfa16 P 6222dc65f238415894ca42c305161c4d: Deleting consensus metadata
I20260807 22:39:35.797930 25152 catalog_manager.cc:4841] TS 6222dc65f238415894ca42c305161c4d (127.28.82.194:37733): tablet 630d03b67e234e22b904e968c6abfa16 (table test-table [id=849d791fe23f423f8eb80ea77e1904bd]) successfully deleted
W20260807 22:39:35.804051 25155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.82.193:40169 (error 108)
I20260807 22:39:35.804287 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 25469
I20260807 22:39:35.806993 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:35.804467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.193:40169 (local address 127.0.0.1:44712)
0807 22:39:35.806882 (+ 2415us) negotiation.cc:107] Waiting for socket to connect
0807 22:39:35.806950 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.193:40169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2383}
W20260807 22:39:35.808861 25155 catalog_manager.cc:4584] TS 351c5d8d9cce4504ad46d3af47bd4433 (127.28.82.193:40169): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 630d03b67e234e22b904e968c6abfa16: Network error: Client connection negotiation failed: client connection to 127.28.82.193:40169: connect: Connection refused (error 111)
I20260807 22:39:35.813853 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 25662
I20260807 22:39:35.823870 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 25127
2026-08-07T22:39:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (865 ms)
[----------] 6 tests from EncryptionPolicyTest (9193 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:35 dist-test-slave-7kzw krb5kdc[25901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:35 dist-test-slave-7kzw krb5kdc[25901](info): set up 2 sockets
Aug 07 22:39:35 dist-test-slave-7kzw krb5kdc[25901](info): commencing operation
krb5kdc: starting...
W20260807 22:39:37.032518 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:37 dist-test-slave-7kzw krb5kdc[25901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:37.157418 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:37.238986 25917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:37.239245 25917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:37.239284 25917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:37.240397 25917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:37.240468 25917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:37.240494 25917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:37.240515 25917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:37.240542 25917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:37.241991 25917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:37.242276 25917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:37.242556 25917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:37.247468 25931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:37.247401 25933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:37.247411 25930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:37.247620 25917 server_base.cc:1034] running on GCE node
I20260807 22:39:37.247923 25917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:37.248154 25917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:37.249317 25917 hybrid_clock.cc:648] HybridClock initialized: now 1786142377249315 us; error 41 us; skew 500 ppm
Aug 07 22:39:37 dist-test-slave-7kzw krb5kdc[25901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:37.257014 25917 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:37.257467 25917 webserver.cc:460] Webserver started at http://127.0.0.1:45209/ using document root <none> and password file <none>
I20260807 22:39:37.257697 25917 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:37.257767 25917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:37.257922 25917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:37.259078 25917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "392f622e057242daa6c4368a6d9e6036"
format_stamp: "Formatted at 2026-08-07 22:39:37 on dist-test-slave-7kzw"
server_key: "1cb6225efadaa2d21792c668197b0609"
server_key_iv: "56b9c0cd500febac38b6fd3198d7a3ef"
server_key_version: "encryptionkey@0"
I20260807 22:39:37.259444 25917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "392f622e057242daa6c4368a6d9e6036"
format_stamp: "Formatted at 2026-08-07 22:39:37 on dist-test-slave-7kzw"
server_key: "1cb6225efadaa2d21792c668197b0609"
server_key_iv: "56b9c0cd500febac38b6fd3198d7a3ef"
server_key_version: "encryptionkey@0"
I20260807 22:39:37.260818 25917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:39:37.261536 25942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:37.261742 25917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:37.261838 25917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "392f622e057242daa6c4368a6d9e6036"
format_stamp: "Formatted at 2026-08-07 22:39:37 on dist-test-slave-7kzw"
server_key: "1cb6225efadaa2d21792c668197b0609"
server_key_iv: "56b9c0cd500febac38b6fd3198d7a3ef"
server_key_version: "encryptionkey@0"
I20260807 22:39:37.261904 25917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:37.274739 25917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:37.277103 25917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:37.277220 25917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:37.281242 25917 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45269
I20260807 22:39:37.281327 26021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45269 every 8 connection(s)
I20260807 22:39:37.281839 25917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:37.282188 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 25917
I20260807 22:39:37.282320 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:37.282627 29003 external_mini_cluster.cc:1428] Setting key 369c0874d0f088f83db8ec4233512c23
I20260807 22:39:37.282738 26022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:37.286099 26022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036: Bootstrap starting.
I20260807 22:39:37.286979 26022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:37.287314 26022 log.cc:824] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:37.288012 26022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036: No bootstrap required, opened a new log
Aug 07 22:39:37 dist-test-slave-7kzw krb5kdc[25901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:39:37.289935 26022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f622e057242daa6c4368a6d9e6036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45269 } }
I20260807 22:39:37.290097 26022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:37.290151 26022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392f622e057242daa6c4368a6d9e6036, State: Initialized, Role: FOLLOWER
I20260807 22:39:37.290297 26022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f622e057242daa6c4368a6d9e6036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45269 } }
I20260807 22:39:37.290402 26022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:37.290443 26022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:37.290529 26022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:37.291258 26022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f622e057242daa6c4368a6d9e6036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45269 } }
I20260807 22:39:37.291369 26022 leader_election.cc:302] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 392f622e057242daa6c4368a6d9e6036; no voters:
I20260807 22:39:37.291529 26022 leader_election.cc:288] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:37.291586 26031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:37.291736 26031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [term 1 LEADER]: Becoming Leader. State: Replica: 392f622e057242daa6c4368a6d9e6036, State: Running, Role: LEADER
I20260807 22:39:37.291790 26022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:37.291851 26031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f622e057242daa6c4368a6d9e6036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45269 } }
I20260807 22:39:37.292160 26031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 392f622e057242daa6c4368a6d9e6036. Latest consensus state: current_term: 1 leader_uuid: "392f622e057242daa6c4368a6d9e6036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f622e057242daa6c4368a6d9e6036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45269 } } }
I20260807 22:39:37.292249 26031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:37.292172 26034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "392f622e057242daa6c4368a6d9e6036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392f622e057242daa6c4368a6d9e6036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45269 } } }
I20260807 22:39:37.292421 26034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036 [sys.catalog]: This master's current role is: LEADER
W20260807 22:39:37.293720 26054 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:39:37.293779 26054 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:39:37.294018 26042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:37.294024 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:37.283241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36431 (local address 127.0.0.1:45269)
0807 22:39:37.283443 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:37.283453 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:37.283487 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:39:37.283891 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:37.283897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:37.283951 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:37.284060 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:37.284879 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:37.285789 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:37.286877 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:37.287352 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:37.287485 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:37.289853 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:37.289858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:37.289858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:37.289871 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:37.291947 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:37.292486 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:37.292489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:37.292489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:37.292611 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:37.292880 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:37.292882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:37.292882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:37.293127 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:37.293347 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:37.293478 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 22:39:37.293693 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260807 22:39:37.294232 26042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:37.295089 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:37.295921 26042 catalog_manager.cc:1347] Generated new cluster ID: 1f5ccf513178479795e768bc32fe2093
I20260807 22:39:37.295974 26042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:37.305995 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:37.296586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60819 (local address 127.0.0.1:45269)
0807 22:39:37.298559 (+ 1973us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:37.298561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:37.298699 (+ 138us) server_negotiation.cc:407] Connection header received
0807 22:39:37.298768 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:37.298769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:37.298779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:37.298828 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:37.299545 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:37.300277 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:37.301272 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:37.301575 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:37.301635 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:37.303049 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:37.303050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:37.303050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:37.303057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:37.304646 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:37.305081 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:37.305082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:37.305082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:37.305138 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:37.305369 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:37.305370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:37.305370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:37.305485 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:37.305577 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:37.305746 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 22:39:37.305815 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1952}
I20260807 22:39:37.309471 26042 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:37.309968 26042 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:37.312753 26042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 392f622e057242daa6c4368a6d9e6036: Generated new TSK 0
I20260807 22:39:37.312954 26042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:37.320817 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 25917
W20260807 22:39:37.333343 25897 connection.cc:535] client connection to 127.0.0.1:45269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45269): Connection reset by peer (error 104)
2026-08-07T22:39:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:37 dist-test-slave-7kzw krb5kdc[26068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:37 dist-test-slave-7kzw krb5kdc[26068](info): set up 2 sockets
Aug 07 22:39:37 dist-test-slave-7kzw krb5kdc[26068](info): commencing operation
krb5kdc: starting...
W20260807 22:39:38.536382 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:38 dist-test-slave-7kzw krb5kdc[26068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:38.663291 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44615
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:38.745338 26086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:38.745543 26086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:38.745566 26086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:38.746771 26086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:38.746815 26086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:38.746829 26086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:38.746841 26086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:38.746893 26086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:38.748428 26086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44615
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:38.748662 26086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:38.748900 26086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:38.754168 26095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:38.754208 26092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:38.754195 26091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:38.754426 26086 server_base.cc:1034] running on GCE node
I20260807 22:39:38.754815 26086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:38.755157 26086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:38.756330 26086 hybrid_clock.cc:648] HybridClock initialized: now 1786142378756305 us; error 41 us; skew 500 ppm
Aug 07 22:39:38 dist-test-slave-7kzw krb5kdc[26068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142378, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:38.759138 26086 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:38.759516 26086 webserver.cc:460] Webserver started at http://127.0.0.1:35087/ using document root <none> and password file <none>
I20260807 22:39:38.759718 26086 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:38.759759 26086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:38.759845 26086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:38.760828 26086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "2a2cef5a2adc4532b62dda1bfb5c150f"
format_stamp: "Formatted at 2026-08-07 22:39:38 on dist-test-slave-7kzw"
server_key: "77df25383a6a21e0ef9e60e877508958"
server_key_iv: "7fefc5edd7e68228d08e645c03dedb33"
server_key_version: "encryptionkey@0"
I20260807 22:39:38.761144 26086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "2a2cef5a2adc4532b62dda1bfb5c150f"
format_stamp: "Formatted at 2026-08-07 22:39:38 on dist-test-slave-7kzw"
server_key: "77df25383a6a21e0ef9e60e877508958"
server_key_iv: "7fefc5edd7e68228d08e645c03dedb33"
server_key_version: "encryptionkey@0"
I20260807 22:39:38.762403 26086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:39:38.763202 26106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:38.763360 26086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:38.763428 26086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "2a2cef5a2adc4532b62dda1bfb5c150f"
format_stamp: "Formatted at 2026-08-07 22:39:38 on dist-test-slave-7kzw"
server_key: "77df25383a6a21e0ef9e60e877508958"
server_key_iv: "7fefc5edd7e68228d08e645c03dedb33"
server_key_version: "encryptionkey@0"
I20260807 22:39:38.763526 26086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:38.792001 26086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:38.794409 26086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:38.794554 26086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:38.798816 26086 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44615
I20260807 22:39:38.798956 26180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44615 every 8 connection(s)
I20260807 22:39:38.799284 26086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:38.800244 26181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:38.802585 26181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f: Bootstrap starting.
I20260807 22:39:38.803186 26181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:38.803483 26181 log.cc:824] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:38.804239 26181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f: No bootstrap required, opened a new log
I20260807 22:39:38.805866 26181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2cef5a2adc4532b62dda1bfb5c150f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } }
I20260807 22:39:38.806000 26181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:38.806025 26181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2cef5a2adc4532b62dda1bfb5c150f, State: Initialized, Role: FOLLOWER
I20260807 22:39:38.806197 26181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2cef5a2adc4532b62dda1bfb5c150f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } }
I20260807 22:39:38.806277 26181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:38.806299 26181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:38.806360 26181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:38.806938 26181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2cef5a2adc4532b62dda1bfb5c150f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } }
I20260807 22:39:38.807025 26181 leader_election.cc:302] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a2cef5a2adc4532b62dda1bfb5c150f; no voters:
I20260807 22:39:38.807207 26181 leader_election.cc:288] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:38.807283 26185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:38.807485 26181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:38.807515 26185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [term 1 LEADER]: Becoming Leader. State: Replica: 2a2cef5a2adc4532b62dda1bfb5c150f, State: Running, Role: LEADER
I20260807 22:39:38.807641 26185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2cef5a2adc4532b62dda1bfb5c150f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } }
I20260807 22:39:38.808343 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 26086
I20260807 22:39:38.808334 26186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a2cef5a2adc4532b62dda1bfb5c150f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2cef5a2adc4532b62dda1bfb5c150f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } } }
I20260807 22:39:38.808422 26186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:38.808460 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:38.808627 26187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a2cef5a2adc4532b62dda1bfb5c150f. Latest consensus state: current_term: 1 leader_uuid: "2a2cef5a2adc4532b62dda1bfb5c150f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2cef5a2adc4532b62dda1bfb5c150f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44615 } } }
I20260807 22:39:38.808693 26187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:38.808710 29003 external_mini_cluster.cc:1428] Setting key 5df50f1210400bcac5b44ac25d7aa372
I20260807 22:39:38.808928 26201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:38.809480 26201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:38.811203 26201 catalog_manager.cc:1347] Generated new cluster ID: d817a6e8dbed4d5d9a172ac8b6c4e3df
I20260807 22:39:38.811306 26201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:38.814806 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:38.809226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39823 (local address 127.0.0.1:44615)
0807 22:39:38.809413 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:38.809424 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:38.809671 (+ 247us) server_negotiation.cc:407] Connection header received
0807 22:39:38.809788 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:38.809795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:38.809850 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:38.809989 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:38.810237 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:38.811578 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:38.812663 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:38.813249 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:38.813341 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:38.814094 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:38.814098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:38.814098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:38.814293 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:38.814329 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:38.814342 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:38.814417 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:38.814575 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 22:39:38.814600 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260807 22:39:38.815814 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:38.821002 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:38.817219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40735 (local address 127.0.0.1:44615)
0807 22:39:38.817250 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:38.817252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:38.817390 (+ 138us) server_negotiation.cc:407] Connection header received
0807 22:39:38.817420 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:38.817421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:38.817432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:38.817509 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:38.817620 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:38.818675 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:38.819641 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:38.819971 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:38.820063 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:38.820744 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:38.820745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:38.820745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:38.820751 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:38.820767 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:38.820772 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:38.820845 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:38.820915 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 22:39:38.820930 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 22:39:38.825448 26201 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:38.826097 26201 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:38.830375 26201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a2cef5a2adc4532b62dda1bfb5c150f: Generated new TSK 0
I20260807 22:39:38.830624 26201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:38.838140 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 26086
W20260807 22:39:38.847041 26065 connection.cc:535] client connection to 127.0.0.1:44615 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44615): Connection reset by peer (error 104)
2026-08-07T22:39:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:38 dist-test-slave-7kzw krb5kdc[26227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:38 dist-test-slave-7kzw krb5kdc[26227](info): set up 2 sockets
Aug 07 22:39:38 dist-test-slave-7kzw krb5kdc[26227](info): commencing operation
krb5kdc: starting...
W20260807 22:39:40.071157 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:40 dist-test-slave-7kzw krb5kdc[26227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:40.197678 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37253
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:40.282549 26254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:40.282809 26254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:40.282847 26254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:40.283989 26254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:40.284061 26254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:40.284085 26254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:40.284119 26254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:40.284169 26254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:40.285681 26254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37253
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:40.285967 26254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:40.286199 26254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:40.291163 26263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:40.291189 26260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:40.291189 26265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:40.291440 26254 server_base.cc:1034] running on GCE node
I20260807 22:39:40.291714 26254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:40.291955 26254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:40.293119 26254 hybrid_clock.cc:648] HybridClock initialized: now 1786142380293099 us; error 43 us; skew 500 ppm
Aug 07 22:39:40 dist-test-slave-7kzw krb5kdc[26227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:40.295954 26254 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:40.296422 26254 webserver.cc:460] Webserver started at http://127.0.0.1:34775/ using document root <none> and password file <none>
I20260807 22:39:40.296675 26254 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:40.296752 26254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:40.296911 26254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:40.298032 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "8de8774dd8e64ec49d0359b74c9604e2"
format_stamp: "Formatted at 2026-08-07 22:39:40 on dist-test-slave-7kzw"
server_key: "f3c87ea9e8d2d0826341885540027291"
server_key_iv: "c076447c7c3dcb5b58554f7fc0fdff09"
server_key_version: "encryptionkey@0"
I20260807 22:39:40.298429 26254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "8de8774dd8e64ec49d0359b74c9604e2"
format_stamp: "Formatted at 2026-08-07 22:39:40 on dist-test-slave-7kzw"
server_key: "f3c87ea9e8d2d0826341885540027291"
server_key_iv: "c076447c7c3dcb5b58554f7fc0fdff09"
server_key_version: "encryptionkey@0"
I20260807 22:39:40.299948 26254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:40.300806 26272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:40.301013 26254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:40.301136 26254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "8de8774dd8e64ec49d0359b74c9604e2"
format_stamp: "Formatted at 2026-08-07 22:39:40 on dist-test-slave-7kzw"
server_key: "f3c87ea9e8d2d0826341885540027291"
server_key_iv: "c076447c7c3dcb5b58554f7fc0fdff09"
server_key_version: "encryptionkey@0"
I20260807 22:39:40.301234 26254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:40.305084 26254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:40.307360 26254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:40.307543 26254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:40.311959 26254 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37253
I20260807 22:39:40.311990 26345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37253 every 8 connection(s)
I20260807 22:39:40.312371 26254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:40.313318 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 26254
I20260807 22:39:40.313459 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:40.313699 29003 external_mini_cluster.cc:1428] Setting key d9e25483c2f8faa8496ba27f6a2858bb
I20260807 22:39:40.313699 26346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:40.316641 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:40.314254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56615 (local address 127.0.0.1:37253)
0807 22:39:40.314456 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:40.314524 (+ 68us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:40.314600 (+ 76us) server_negotiation.cc:407] Connection header received
0807 22:39:40.315242 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:40.315249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:40.315276 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:40.315396 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:40.316135 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:40.316139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:40.316139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:40.316143 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:40.316182 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:40.316193 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:40.316256 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:40.316351 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 22:39:40.316411 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260807 22:39:40.317107 26346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2: Bootstrap starting.
I20260807 22:39:40.317622 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:40.317817 26346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:40.318121 26346 log.cc:824] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:40.318996 26346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2: No bootstrap required, opened a new log
I20260807 22:39:40.320389 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:40.319297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56767 (local address 127.0.0.1:37253)
0807 22:39:40.319332 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:40.319334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:40.319453 (+ 119us) server_negotiation.cc:407] Connection header received
0807 22:39:40.319528 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:40.319529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:40.319534 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:40.319578 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:40.320186 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:40.320187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:40.320188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:40.320192 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:40.320206 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:40.320210 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:40.320259 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:40.320312 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 22:39:40.320325 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 22:39:40.321084 26346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de8774dd8e64ec49d0359b74c9604e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } }
I20260807 22:39:40.321220 26346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:40.321244 26346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de8774dd8e64ec49d0359b74c9604e2, State: Initialized, Role: FOLLOWER
I20260807 22:39:40.321388 26346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de8774dd8e64ec49d0359b74c9604e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } }
I20260807 22:39:40.321462 26346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:40.321520 26346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:40.321573 26346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:40.322103 26346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de8774dd8e64ec49d0359b74c9604e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } }
I20260807 22:39:40.322219 26346 leader_election.cc:302] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8de8774dd8e64ec49d0359b74c9604e2; no voters:
I20260807 22:39:40.322404 26346 leader_election.cc:288] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:40.322569 26358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:40.322762 26358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [term 1 LEADER]: Becoming Leader. State: Replica: 8de8774dd8e64ec49d0359b74c9604e2, State: Running, Role: LEADER
I20260807 22:39:40.322774 26346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:40.322902 26358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de8774dd8e64ec49d0359b74c9604e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } }
I20260807 22:39:40.323276 26359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8de8774dd8e64ec49d0359b74c9604e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de8774dd8e64ec49d0359b74c9604e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } } }
I20260807 22:39:40.323403 26359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:40.323280 26361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8de8774dd8e64ec49d0359b74c9604e2. Latest consensus state: current_term: 1 leader_uuid: "8de8774dd8e64ec49d0359b74c9604e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de8774dd8e64ec49d0359b74c9604e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37253 } } }
I20260807 22:39:40.323556 26361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:40.323911 26371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:40.324390 26371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:40.326028 26371 catalog_manager.cc:1347] Generated new cluster ID: 3497c878267a41db924e6ba33fc71342
I20260807 22:39:40.326103 26371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:40.332048 26371 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:40.332218 26371 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:40.339416 26371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8de8774dd8e64ec49d0359b74c9604e2: Generated new TSK 0
I20260807 22:39:40.339599 26371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:40.347556 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 26254
2026-08-07T22:39:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:40 dist-test-slave-7kzw krb5kdc[26397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:40 dist-test-slave-7kzw krb5kdc[26397](info): set up 2 sockets
Aug 07 22:39:40 dist-test-slave-7kzw krb5kdc[26397](info): commencing operation
krb5kdc: starting...
W20260807 22:39:41.575114 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:41 dist-test-slave-7kzw krb5kdc[26397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:41.701525 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:41.784911 26412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:41.785152 26412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:41.785192 26412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:41.786402 26412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:41.786476 26412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:41.786517 26412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:41.786549 26412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:41.786599 26412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:41.788079 26412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:41.788370 26412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:41.788602 26412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:41.793566 26425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:41.793593 26420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:41.793782 26412 server_base.cc:1034] running on GCE node
W20260807 22:39:41.793656 26421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:41.794139 26412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:41.794440 26412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:41.795707 26412 hybrid_clock.cc:648] HybridClock initialized: now 1786142381795668 us; error 50 us; skew 500 ppm
Aug 07 22:39:41 dist-test-slave-7kzw krb5kdc[26397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:41.798411 26412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:41.798935 26412 webserver.cc:460] Webserver started at http://127.0.0.1:43181/ using document root <none> and password file <none>
I20260807 22:39:41.799165 26412 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:41.799234 26412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:41.799392 26412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:41.800421 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "c2b150c95b8e440f8424c73a080c12fd"
format_stamp: "Formatted at 2026-08-07 22:39:41 on dist-test-slave-7kzw"
server_key: "e607b6dee8db26b30a3151410fa0e8d4"
server_key_iv: "d460341f03f80ed5c4e6cc013e41d648"
server_key_version: "encryptionkey@0"
I20260807 22:39:41.800800 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "c2b150c95b8e440f8424c73a080c12fd"
format_stamp: "Formatted at 2026-08-07 22:39:41 on dist-test-slave-7kzw"
server_key: "e607b6dee8db26b30a3151410fa0e8d4"
server_key_iv: "d460341f03f80ed5c4e6cc013e41d648"
server_key_version: "encryptionkey@0"
I20260807 22:39:41.802132 26412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 22:39:41.802903 26433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:41.803066 26412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:41.803174 26412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "c2b150c95b8e440f8424c73a080c12fd"
format_stamp: "Formatted at 2026-08-07 22:39:41 on dist-test-slave-7kzw"
server_key: "e607b6dee8db26b30a3151410fa0e8d4"
server_key_iv: "d460341f03f80ed5c4e6cc013e41d648"
server_key_version: "encryptionkey@0"
I20260807 22:39:41.803252 26412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:41.814965 26412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:41.817284 26412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:41.817444 26412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:41.821604 26412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34981
I20260807 22:39:41.821638 26501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34981 every 8 connection(s)
I20260807 22:39:41.821985 26412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:41.822940 26503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:41.825174 26503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd: Bootstrap starting.
I20260807 22:39:41.825799 26503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:41.826105 26503 log.cc:824] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:41.826246 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 26412
I20260807 22:39:41.826391 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:41.826598 29003 external_mini_cluster.cc:1428] Setting key cc2d9cf4c2f10c99201b7b6b258ac2fe
I20260807 22:39:41.826974 26503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd: No bootstrap required, opened a new log
I20260807 22:39:41.828663 26503 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b150c95b8e440f8424c73a080c12fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } }
I20260807 22:39:41.828831 26503 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:41.828919 26503 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b150c95b8e440f8424c73a080c12fd, State: Initialized, Role: FOLLOWER
I20260807 22:39:41.829087 26503 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b150c95b8e440f8424c73a080c12fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } }
I20260807 22:39:41.829190 26503 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:41.829236 26503 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:41.829284 26503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:41.830158 26503 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b150c95b8e440f8424c73a080c12fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } }
I20260807 22:39:41.830286 26503 leader_election.cc:302] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2b150c95b8e440f8424c73a080c12fd; no voters:
I20260807 22:39:41.830489 26503 leader_election.cc:288] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:41.830520 26508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:41.830636 26508 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [term 1 LEADER]: Becoming Leader. State: Replica: c2b150c95b8e440f8424c73a080c12fd, State: Running, Role: LEADER
I20260807 22:39:41.830768 26503 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:41.830771 26508 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b150c95b8e440f8424c73a080c12fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } }
I20260807 22:39:41.831159 26508 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2b150c95b8e440f8424c73a080c12fd. Latest consensus state: current_term: 1 leader_uuid: "c2b150c95b8e440f8424c73a080c12fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b150c95b8e440f8424c73a080c12fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } } }
I20260807 22:39:41.831173 26509 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2b150c95b8e440f8424c73a080c12fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2b150c95b8e440f8424c73a080c12fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34981 } } }
I20260807 22:39:41.831326 26508 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:41.831404 26509 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:41.831667 26519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:41.832186 26519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 22:39:41 dist-test-slave-7kzw krb5kdc[26397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:39:41.835476 26519 catalog_manager.cc:1347] Generated new cluster ID: 71bce21f58394285ab6cae42dd5a01d1
I20260807 22:39:41.835537 26519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:41.839128 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:41.827100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33467 (local address 127.0.0.1:34981)
0807 22:39:41.827311 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:41.827321 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:41.827416 (+ 95us) server_negotiation.cc:407] Connection header received
0807 22:39:41.828665 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:41.828673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:41.828726 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:41.828895 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:41.829683 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:41.830817 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:41.831921 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:41.832707 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:41.833146 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:41.835648 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:41.835651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:41.835652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:41.835662 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:41.837329 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:41.837721 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:41.837724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:41.837724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:41.837842 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:41.838070 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:41.838071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:41.838072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:41.838311 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:41.838507 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:41.838642 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 22:39:41.838786 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260807 22:39:41.840160 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:41.857082 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:41.841528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43761 (local address 127.0.0.1:34981)
0807 22:39:41.841566 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:41.841568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:41.850707 (+ 9139us) server_negotiation.cc:407] Connection header received
0807 22:39:41.850785 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:41.850786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:41.850798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:41.850873 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:41.851398 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:41.851884 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:41.852577 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:41.852801 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:41.852881 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:41.854052 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:41.854054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:41.854054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:41.854062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:41.855520 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:41.855900 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:41.855901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:41.855901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:41.855961 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:41.856169 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:41.856170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:41.856171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:41.856281 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:41.856920 (+ 639us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:41.856957 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 22:39:41.857020 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:39:41.864565 26519 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:41.865279 26519 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:41.872658 26519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2b150c95b8e440f8424c73a080c12fd: Generated new TSK 0
I20260807 22:39:41.872834 26519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:41.876196 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 26412
2026-08-07T22:39:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:41 dist-test-slave-7kzw krb5kdc[26551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:41 dist-test-slave-7kzw krb5kdc[26551](info): set up 2 sockets
Aug 07 22:39:41 dist-test-slave-7kzw krb5kdc[26551](info): commencing operation
krb5kdc: starting...
W20260807 22:39:43.118280 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:43 dist-test-slave-7kzw krb5kdc[26551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:43.239611 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:43.321624 26569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:43.321866 26569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:43.321904 26569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:43.323058 26569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:43.323128 26569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:43.323151 26569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:43.323170 26569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:43.323205 26569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:43.324685 26569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:43.324965 26569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:43.325191 26569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:43.330217 26575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:43.330294 26569 server_base.cc:1034] running on GCE node
W20260807 22:39:43.330253 26578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:43.330247 26576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:43.330762 26569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:43.331107 26569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:43.332254 26569 hybrid_clock.cc:648] HybridClock initialized: now 1786142383332248 us; error 53 us; skew 500 ppm
Aug 07 22:39:43 dist-test-slave-7kzw krb5kdc[26551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:43.339797 26569 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:43.340291 26569 webserver.cc:460] Webserver started at http://127.0.0.1:39843/ using document root <none> and password file <none>
I20260807 22:39:43.340529 26569 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:43.340574 26569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:43.340740 26569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:43.341825 26569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "8dfbaad53561433ab55e2f9b81e350a4"
format_stamp: "Formatted at 2026-08-07 22:39:43 on dist-test-slave-7kzw"
server_key: "7542d8bb907c857c9a898f1912dd1047"
server_key_iv: "94210d040d477a2070d337b2fcf667f5"
server_key_version: "encryptionkey@0"
I20260807 22:39:43.342197 26569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "8dfbaad53561433ab55e2f9b81e350a4"
format_stamp: "Formatted at 2026-08-07 22:39:43 on dist-test-slave-7kzw"
server_key: "7542d8bb907c857c9a898f1912dd1047"
server_key_iv: "94210d040d477a2070d337b2fcf667f5"
server_key_version: "encryptionkey@0"
I20260807 22:39:43.343626 26569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:43.344458 26586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:43.344679 26569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:43.344805 26569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "8dfbaad53561433ab55e2f9b81e350a4"
format_stamp: "Formatted at 2026-08-07 22:39:43 on dist-test-slave-7kzw"
server_key: "7542d8bb907c857c9a898f1912dd1047"
server_key_iv: "94210d040d477a2070d337b2fcf667f5"
server_key_version: "encryptionkey@0"
I20260807 22:39:43.344884 26569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:43.367776 26569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:43.369653 26569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:43.369831 26569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:43.374141 26569 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35417
I20260807 22:39:43.374179 26663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35417 every 8 connection(s)
I20260807 22:39:43.374619 26569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:43.375510 26664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:43.377797 26664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4: Bootstrap starting.
I20260807 22:39:43.378333 26664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:43.378610 26664 log.cc:824] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:43.379289 26664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4: No bootstrap required, opened a new log
I20260807 22:39:43.380867 26664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dfbaad53561433ab55e2f9b81e350a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260807 22:39:43.381006 26664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:43.381031 26664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dfbaad53561433ab55e2f9b81e350a4, State: Initialized, Role: FOLLOWER
I20260807 22:39:43.381178 26664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dfbaad53561433ab55e2f9b81e350a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260807 22:39:43.381263 26664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:43.381296 26664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:43.381327 26664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:43.381840 26664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dfbaad53561433ab55e2f9b81e350a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260807 22:39:43.381932 26664 leader_election.cc:302] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dfbaad53561433ab55e2f9b81e350a4; no voters:
I20260807 22:39:43.382066 26664 leader_election.cc:288] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:43.382151 26667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:43.382340 26664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:43.382324 26667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [term 1 LEADER]: Becoming Leader. State: Replica: 8dfbaad53561433ab55e2f9b81e350a4, State: Running, Role: LEADER
I20260807 22:39:43.382543 26667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dfbaad53561433ab55e2f9b81e350a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260807 22:39:43.382952 26670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dfbaad53561433ab55e2f9b81e350a4. Latest consensus state: current_term: 1 leader_uuid: "8dfbaad53561433ab55e2f9b81e350a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dfbaad53561433ab55e2f9b81e350a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } } }
I20260807 22:39:43.382980 26669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dfbaad53561433ab55e2f9b81e350a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dfbaad53561433ab55e2f9b81e350a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } } }
I20260807 22:39:43.383108 26669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:43.383351 26670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:43.383524 26672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:43.384053 26672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:43.384872 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 26569
I20260807 22:39:43.385007 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:43.385262 29003 external_mini_cluster.cc:1428] Setting key 5f68f291ba56af56b0a3a53338f73a6d
I20260807 22:39:43.386023 26672 catalog_manager.cc:1347] Generated new cluster ID: 40ef9647a6434938b1336be47c854923
I20260807 22:39:43.386111 26672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:43.391698 26689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:43.385763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54863 (local address 127.0.0.1:35417)
0807 22:39:43.385938 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:43.385964 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:43.386278 (+ 314us) server_negotiation.cc:407] Connection header received
0807 22:39:43.386408 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:43.386416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:43.386503 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:43.386640 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:43.386889 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:43.388342 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:43.389432 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:43.390064 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:43.390179 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:43.390921 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:43.390925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:43.390926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:43.391143 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:43.391185 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:43.391211 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:43.391286 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:43.391353 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 22:39:43.391431 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260807 22:39:43.392627 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:43.397593 26689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:43.394138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43923 (local address 127.0.0.1:35417)
0807 22:39:43.394169 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:43.394171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:43.394314 (+ 143us) server_negotiation.cc:407] Connection header received
0807 22:39:43.394372 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:43.394373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:43.394383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:43.394431 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:43.394594 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:43.395295 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:43.396326 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:43.396615 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:43.396671 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:43.397356 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:43.397358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:43.397358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:43.397364 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:43.397378 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:43.397382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:43.397436 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:43.397500 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 22:39:43.397521 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 22:39:43.411964 26672 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:43.412477 26672 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:43.422672 26672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dfbaad53561433ab55e2f9b81e350a4: Generated new TSK 0
I20260807 22:39:43.422876 26672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:43.425892 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 26569
2026-08-07T22:39:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:43 dist-test-slave-7kzw krb5kdc[26700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:43 dist-test-slave-7kzw krb5kdc[26700](info): set up 2 sockets
Aug 07 22:39:43 dist-test-slave-7kzw krb5kdc[26700](info): commencing operation
krb5kdc: starting...
W20260807 22:39:44.649873 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:44 dist-test-slave-7kzw krb5kdc[26700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:44.773023 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:44.857105 26721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:44.857297 26721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:44.857319 26721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:44.858554 26721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:44.858597 26721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:44.858611 26721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:44.858624 26721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:44.858636 26721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:44.860090 26721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:44.860342 26721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:44.860587 26721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:44.865679 26729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:44.865665 26733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:44.865701 26730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:44.865990 26721 server_base.cc:1034] running on GCE node
I20260807 22:39:44.866271 26721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:44.866607 26721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:44.867760 26721 hybrid_clock.cc:648] HybridClock initialized: now 1786142384867750 us; error 32 us; skew 500 ppm
Aug 07 22:39:44 dist-test-slave-7kzw krb5kdc[26700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142384, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:44.870229 26721 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:44.870743 26721 webserver.cc:460] Webserver started at http://127.0.0.1:33051/ using document root <none> and password file <none>
I20260807 22:39:44.870990 26721 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:44.871066 26721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:44.871213 26721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:44.872243 26721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "ba7e43af4d304a70a43b84723c3cbad9"
format_stamp: "Formatted at 2026-08-07 22:39:44 on dist-test-slave-7kzw"
server_key: "b6aaf5cd6f4c07127c7c52ba3bc978e2"
server_key_iv: "7e167c1e60284a1c761c9c1bda1eb65c"
server_key_version: "encryptionkey@0"
I20260807 22:39:44.872610 26721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "ba7e43af4d304a70a43b84723c3cbad9"
format_stamp: "Formatted at 2026-08-07 22:39:44 on dist-test-slave-7kzw"
server_key: "b6aaf5cd6f4c07127c7c52ba3bc978e2"
server_key_iv: "7e167c1e60284a1c761c9c1bda1eb65c"
server_key_version: "encryptionkey@0"
I20260807 22:39:44.873932 26721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:44.874883 26745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:44.875101 26721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:44.875214 26721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "ba7e43af4d304a70a43b84723c3cbad9"
format_stamp: "Formatted at 2026-08-07 22:39:44 on dist-test-slave-7kzw"
server_key: "b6aaf5cd6f4c07127c7c52ba3bc978e2"
server_key_iv: "7e167c1e60284a1c761c9c1bda1eb65c"
server_key_version: "encryptionkey@0"
I20260807 22:39:44.875293 26721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:44.878614 26721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:44.880520 26721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:44.880688 26721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:44.884873 26721 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42117
I20260807 22:39:44.884905 26822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42117 every 8 connection(s)
I20260807 22:39:44.885229 26721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:44.886302 26823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:44.888183 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 26721
I20260807 22:39:44.888327 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:44.888545 29003 external_mini_cluster.cc:1428] Setting key 9c80dfe745662d385656789011e352c8
I20260807 22:39:44.888717 26823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9: Bootstrap starting.
I20260807 22:39:44.889480 26823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:44.889840 26823 log.cc:824] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:44.890854 26823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9: No bootstrap required, opened a new log
I20260807 22:39:44.891477 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:44.889095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43089 (local address 127.0.0.1:42117)
0807 22:39:44.889347 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:44.889356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:44.889389 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:39:44.890021 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:44.890030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:44.890064 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:44.890197 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:44.891048 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:44.891051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:44.891052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:44.891056 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:44.891104 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:44.891114 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:44.891214 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:44.891271 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 22:39:44.891311 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":110,"threads_started":1}
I20260807 22:39:44.892556 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:44.893270 26823 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba7e43af4d304a70a43b84723c3cbad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42117 } }
I20260807 22:39:44.893419 26823 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:44.893499 26823 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7e43af4d304a70a43b84723c3cbad9, State: Initialized, Role: FOLLOWER
I20260807 22:39:44.893633 26823 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba7e43af4d304a70a43b84723c3cbad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42117 } }
I20260807 22:39:44.893747 26823 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:44.893793 26823 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:44.893842 26823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:44.894449 26823 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba7e43af4d304a70a43b84723c3cbad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42117 } }
I20260807 22:39:44.894596 26823 leader_election.cc:302] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba7e43af4d304a70a43b84723c3cbad9; no voters:
I20260807 22:39:44.894757 26823 leader_election.cc:288] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:44.894904 26838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:44.895033 26823 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:44.895118 26838 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [term 1 LEADER]: Becoming Leader. State: Replica: ba7e43af4d304a70a43b84723c3cbad9, State: Running, Role: LEADER
I20260807 22:39:44.895335 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:44.894034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51639 (local address 127.0.0.1:42117)
0807 22:39:44.894099 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:44.894102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:44.894225 (+ 123us) server_negotiation.cc:407] Connection header received
0807 22:39:44.894303 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:44.894304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:44.894308 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:44.894362 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:44.895000 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:44.895001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:44.895002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:44.895006 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:44.895020 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:44.895024 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:44.895068 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:44.895214 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 22:39:44.895230 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:39:44.895262 26838 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba7e43af4d304a70a43b84723c3cbad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42117 } }
I20260807 22:39:44.895813 26843 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba7e43af4d304a70a43b84723c3cbad9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba7e43af4d304a70a43b84723c3cbad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42117 } } }
I20260807 22:39:44.895917 26843 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:44.895833 26845 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba7e43af4d304a70a43b84723c3cbad9. Latest consensus state: current_term: 1 leader_uuid: "ba7e43af4d304a70a43b84723c3cbad9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba7e43af4d304a70a43b84723c3cbad9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42117 } } }
I20260807 22:39:44.896147 26854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:44.896189 26845 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:44.896637 26854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:44.898221 26854 catalog_manager.cc:1347] Generated new cluster ID: 2131ac542f4a44c5b2d39232f9909099
I20260807 22:39:44.898283 26854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:44.908463 26854 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:44.908629 26854 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:44.915733 26854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba7e43af4d304a70a43b84723c3cbad9: Generated new TSK 0
I20260807 22:39:44.915908 26854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:44.918615 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 26721
2026-08-07T22:39:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:44 dist-test-slave-7kzw krb5kdc[26869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:44 dist-test-slave-7kzw krb5kdc[26869](info): set up 2 sockets
Aug 07 22:39:44 dist-test-slave-7kzw krb5kdc[26869](info): commencing operation
krb5kdc: starting...
W20260807 22:39:46.143415 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:46 dist-test-slave-7kzw krb5kdc[26869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:46.264969 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:46.346910 26886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:46.347117 26886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:46.347141 26886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:46.348278 26886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:46.348322 26886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:46.348336 26886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:46.348349 26886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:46.348361 26886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:46.349833 26886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:46.350085 26886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:46.350312 26886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:46.355320 26896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:46.355319 26891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:46.355389 26892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:46.355687 26886 server_base.cc:1034] running on GCE node
I20260807 22:39:46.355940 26886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:46.356184 26886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:46.357329 26886 hybrid_clock.cc:648] HybridClock initialized: now 1786142386357318 us; error 30 us; skew 500 ppm
Aug 07 22:39:46 dist-test-slave-7kzw krb5kdc[26869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:46.359977 26886 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:46.360458 26886 webserver.cc:460] Webserver started at http://127.0.0.1:37289/ using document root <none> and password file <none>
I20260807 22:39:46.360698 26886 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:46.360766 26886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:46.360915 26886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:46.361946 26886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "dca062fa423f4f5487dd80d24303a5b8"
format_stamp: "Formatted at 2026-08-07 22:39:46 on dist-test-slave-7kzw"
server_key: "5354648e62275eed9cf2d565a54d29b6"
server_key_iv: "acb888386473c13305d80a3ae8d76c4c"
server_key_version: "encryptionkey@0"
I20260807 22:39:46.362315 26886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "dca062fa423f4f5487dd80d24303a5b8"
format_stamp: "Formatted at 2026-08-07 22:39:46 on dist-test-slave-7kzw"
server_key: "5354648e62275eed9cf2d565a54d29b6"
server_key_iv: "acb888386473c13305d80a3ae8d76c4c"
server_key_version: "encryptionkey@0"
I20260807 22:39:46.363695 26886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:46.364465 26906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:46.364645 26886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:39:46.364748 26886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "dca062fa423f4f5487dd80d24303a5b8"
format_stamp: "Formatted at 2026-08-07 22:39:46 on dist-test-slave-7kzw"
server_key: "5354648e62275eed9cf2d565a54d29b6"
server_key_iv: "acb888386473c13305d80a3ae8d76c4c"
server_key_version: "encryptionkey@0"
I20260807 22:39:46.364830 26886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:46.403054 26886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:46.405469 26886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:46.405634 26886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:46.410104 26886 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39919
I20260807 22:39:46.410166 26972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39919 every 8 connection(s)
I20260807 22:39:46.410593 26886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:46.411515 26973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:46.413796 26973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8: Bootstrap starting.
I20260807 22:39:46.414381 26973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:46.414727 26973 log.cc:824] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:46.415426 26973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8: No bootstrap required, opened a new log
I20260807 22:39:46.417025 26973 raft_consensus.cc:348] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca062fa423f4f5487dd80d24303a5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260807 22:39:46.417162 26973 raft_consensus.cc:374] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:46.417186 26973 raft_consensus.cc:729] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca062fa423f4f5487dd80d24303a5b8, State: Initialized, Role: FOLLOWER
I20260807 22:39:46.417379 26973 consensus_queue.cc:260] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca062fa423f4f5487dd80d24303a5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260807 22:39:46.417472 26973 raft_consensus.cc:388] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:46.417496 26973 raft_consensus.cc:482] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:46.417568 26973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:46.418149 26973 raft_consensus.cc:504] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca062fa423f4f5487dd80d24303a5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260807 22:39:46.418263 26973 leader_election.cc:302] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dca062fa423f4f5487dd80d24303a5b8; no voters:
I20260807 22:39:46.418453 26973 leader_election.cc:288] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:46.418666 26977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:46.418752 26973 sys_catalog.cc:564] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:46.418840 26977 raft_consensus.cc:686] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [term 1 LEADER]: Becoming Leader. State: Replica: dca062fa423f4f5487dd80d24303a5b8, State: Running, Role: LEADER
I20260807 22:39:46.418934 26977 consensus_queue.cc:237] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca062fa423f4f5487dd80d24303a5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260807 22:39:46.419540 26978 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dca062fa423f4f5487dd80d24303a5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca062fa423f4f5487dd80d24303a5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } } }
I20260807 22:39:46.419516 26979 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dca062fa423f4f5487dd80d24303a5b8. Latest consensus state: current_term: 1 leader_uuid: "dca062fa423f4f5487dd80d24303a5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca062fa423f4f5487dd80d24303a5b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } } }
I20260807 22:39:46.419647 26978 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:46.419656 26979 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:46.419873 26986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:46.420210 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 26886
I20260807 22:39:46.420359 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:46.420428 26986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:46.420590 29003 external_mini_cluster.cc:1428] Setting key 797e4ea4480d74c7b6d8ff4f8f67039c
I20260807 22:39:46.422103 26986 catalog_manager.cc:1347] Generated new cluster ID: 9254284b682a4a7290ee9aee6187701a
I20260807 22:39:46.422163 26986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:39:46 dist-test-slave-7kzw krb5kdc[26869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:39:46.430631 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:46.421117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:40545 (local address 127.0.0.1:39919)
0807 22:39:46.421279 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:46.421290 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:46.421372 (+ 82us) server_negotiation.cc:407] Connection header received
0807 22:39:46.421547 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:46.421555 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:46.421638 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:46.421781 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:46.422585 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:46.423535 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:46.424298 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:46.424712 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:46.424854 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:46.427299 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:46.427303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:46.427303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:46.427315 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:46.428983 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:46.429405 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:46.429413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:46.429413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:46.429511 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:46.429751 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:46.429752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:46.429752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:46.429988 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:46.430177 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:46.430263 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 22:39:46.430375 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260807 22:39:46.431635 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:46.439221 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:46.433006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:38531 (local address 127.0.0.1:39919)
0807 22:39:46.433043 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:46.433045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:46.433161 (+ 116us) server_negotiation.cc:407] Connection header received
0807 22:39:46.433240 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:46.433241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:46.433250 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:46.433299 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:46.433809 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:46.434295 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:46.435052 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:46.435273 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:46.435361 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:46.436547 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:46.436548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:46.436549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:46.436556 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:46.437993 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:46.438388 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:46.438389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:46.438390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:46.438453 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:46.438711 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:46.438712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:46.438713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:46.438823 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:46.438933 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:46.439057 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 22:39:46.439119 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":18}
I20260807 22:39:46.453668 26986 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:46.454371 26986 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:46.459823 26986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dca062fa423f4f5487dd80d24303a5b8: Generated new TSK 0
I20260807 22:39:46.459998 26986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:46.462987 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 26886
W20260807 22:39:46.476454 26864 connection.cc:535] client connection to 127.0.0.1:39919 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39919): Connection reset by peer (error 104)
2026-08-07T22:39:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:46 dist-test-slave-7kzw krb5kdc[27012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:46 dist-test-slave-7kzw krb5kdc[27012](info): set up 2 sockets
Aug 07 22:39:46 dist-test-slave-7kzw krb5kdc[27012](info): commencing operation
krb5kdc: starting...
W20260807 22:39:47.684628 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:47 dist-test-slave-7kzw krb5kdc[27012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:47.810232 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:47.891222 27033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:47.891466 27033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:47.891505 27033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:47.892642 27033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:47.892711 27033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:47.892735 27033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:47.892763 27033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:47.892793 27033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:47.894248 27033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:47.894560 27033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:47.894791 27033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:47.899621 27039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:47.899693 27040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:47.899744 27043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:47.899883 27033 server_base.cc:1034] running on GCE node
I20260807 22:39:47.900120 27033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:47.900416 27033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:47.901607 27033 hybrid_clock.cc:648] HybridClock initialized: now 1786142387901588 us; error 28 us; skew 500 ppm
Aug 07 22:39:47 dist-test-slave-7kzw krb5kdc[27012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:47.904211 27033 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:47.904668 27033 webserver.cc:460] Webserver started at http://127.0.0.1:37315/ using document root <none> and password file <none>
I20260807 22:39:47.904929 27033 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:47.905004 27033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:47.905153 27033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:47.906162 27033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "7d2d21b496cb486a834fb5b2dd1bbc18"
format_stamp: "Formatted at 2026-08-07 22:39:47 on dist-test-slave-7kzw"
server_key: "7c52ba6616f8833c63fe23bbf1745f82"
server_key_iv: "3e145f4119d61fdf622f85f0339a461c"
server_key_version: "encryptionkey@0"
I20260807 22:39:47.906600 27033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "7d2d21b496cb486a834fb5b2dd1bbc18"
format_stamp: "Formatted at 2026-08-07 22:39:47 on dist-test-slave-7kzw"
server_key: "7c52ba6616f8833c63fe23bbf1745f82"
server_key_iv: "3e145f4119d61fdf622f85f0339a461c"
server_key_version: "encryptionkey@0"
I20260807 22:39:47.907969 27033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:47.908783 27054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:47.909039 27033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:47.909145 27033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "7d2d21b496cb486a834fb5b2dd1bbc18"
format_stamp: "Formatted at 2026-08-07 22:39:47 on dist-test-slave-7kzw"
server_key: "7c52ba6616f8833c63fe23bbf1745f82"
server_key_iv: "3e145f4119d61fdf622f85f0339a461c"
server_key_version: "encryptionkey@0"
I20260807 22:39:47.909230 27033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:47.935837 27033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:47.938294 27033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:47.938482 27033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:47.943286 27033 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37127
I20260807 22:39:47.943322 27128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37127 every 8 connection(s)
I20260807 22:39:47.943704 27033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:47.944592 27129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:47.945472 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 27033
I20260807 22:39:47.945611 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:47.945843 29003 external_mini_cluster.cc:1428] Setting key 5678904c3cd2a91649d40991db5e75a8
I20260807 22:39:47.947887 27129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18: Bootstrap starting.
I20260807 22:39:47.948506 27129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:47.948853 27129 log.cc:824] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:47.949656 27129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18: No bootstrap required, opened a new log
I20260807 22:39:47.951596 27133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:47.946397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:44993 (local address 127.0.0.1:37127)
0807 22:39:47.946611 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:47.946621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:47.946655 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:39:47.947120 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:47.947127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:47.947176 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:47.947333 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:47.947580 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:47.948520 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:47.949645 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:47.950101 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:47.950184 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:47.950941 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:47.950946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:47.950946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:47.951111 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:47.951139 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:47.951150 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:47.951215 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:47.951339 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 22:39:47.951384 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260807 22:39:47.952165 27129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2d21b496cb486a834fb5b2dd1bbc18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260807 22:39:47.952364 27129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:47.952423 27129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d2d21b496cb486a834fb5b2dd1bbc18, State: Initialized, Role: FOLLOWER
I20260807 22:39:47.952541 27129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2d21b496cb486a834fb5b2dd1bbc18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260807 22:39:47.952634 27129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:47.952646 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:47.952687 27129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:47.952736 27129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:47.953286 27129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2d21b496cb486a834fb5b2dd1bbc18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260807 22:39:47.953373 27129 leader_election.cc:302] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d2d21b496cb486a834fb5b2dd1bbc18; no voters:
I20260807 22:39:47.953554 27129 leader_election.cc:288] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:47.953588 27136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:47.953724 27136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [term 1 LEADER]: Becoming Leader. State: Replica: 7d2d21b496cb486a834fb5b2dd1bbc18, State: Running, Role: LEADER
I20260807 22:39:47.953809 27136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2d21b496cb486a834fb5b2dd1bbc18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } }
I20260807 22:39:47.953848 27129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:47.954401 27136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d2d21b496cb486a834fb5b2dd1bbc18. Latest consensus state: current_term: 1 leader_uuid: "7d2d21b496cb486a834fb5b2dd1bbc18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2d21b496cb486a834fb5b2dd1bbc18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } } }
I20260807 22:39:47.954526 27136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:47.954690 27143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d2d21b496cb486a834fb5b2dd1bbc18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2d21b496cb486a834fb5b2dd1bbc18" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37127 } } }
I20260807 22:39:47.954767 27143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:47.955061 27153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:47.955555 27153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:47.957935 27153 catalog_manager.cc:1347] Generated new cluster ID: 12d1b71d784948eab9bb5b581638c6a1
I20260807 22:39:47.958001 27153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:47.958614 27133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:47.953991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:60831 (local address 127.0.0.1:37127)
0807 22:39:47.954027 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:47.954029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:47.954176 (+ 147us) server_negotiation.cc:407] Connection header received
0807 22:39:47.954252 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:47.954253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:47.954261 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:47.954302 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:47.954618 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:47.955488 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:47.956556 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:47.956866 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:47.956945 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:47.957580 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:47.957581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:47.957582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:47.957587 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:47.957602 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:47.957606 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:47.957664 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:47.958535 (+ 871us) server_negotiation.cc:299] Negotiation successful
0807 22:39:47.958546 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 22:39:47.978506 27153 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:47.979264 27153 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:47.986423 27153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d2d21b496cb486a834fb5b2dd1bbc18: Generated new TSK 0
I20260807 22:39:47.986627 27153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:47.996196 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 27033
W20260807 22:39:48.008816 27009 connection.cc:535] client connection to 127.0.0.1:37127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37127): Connection reset by peer (error 104)
2026-08-07T22:39:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:48 dist-test-slave-7kzw krb5kdc[27174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:48 dist-test-slave-7kzw krb5kdc[27174](info): set up 2 sockets
Aug 07 22:39:48 dist-test-slave-7kzw krb5kdc[27174](info): commencing operation
krb5kdc: starting...
W20260807 22:39:49.210454 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:49 dist-test-slave-7kzw krb5kdc[27174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:49.333546 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:49.415642 27195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:49.415932 27195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:49.415958 27195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:49.417085 27195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:49.417129 27195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:49.417145 27195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:49.417199 27195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:49.417233 27195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:49.418715 27195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:49.418985 27195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:49.419238 27195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:39:49.424283 27195 server_base.cc:1034] running on GCE node
W20260807 22:39:49.424263 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:49.424207 27206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:49.424186 27204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:49.424711 27195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:49.424953 27195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:49.426092 27195 hybrid_clock.cc:648] HybridClock initialized: now 1786142389426073 us; error 41 us; skew 500 ppm
Aug 07 22:39:49 dist-test-slave-7kzw krb5kdc[27174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:49.431890 27195 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:49.432344 27195 webserver.cc:460] Webserver started at http://127.0.0.1:38337/ using document root <none> and password file <none>
I20260807 22:39:49.432570 27195 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:49.432615 27195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:49.432776 27195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:49.433840 27195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "ee3c3cc0553d46dcbc8aac28fcee5601"
format_stamp: "Formatted at 2026-08-07 22:39:49 on dist-test-slave-7kzw"
server_key: "db2551a1879ce4436c22df66028d6d20"
server_key_iv: "e3c0642fd8f1c3ee65d0b71a6d6d6270"
server_key_version: "encryptionkey@0"
I20260807 22:39:49.434204 27195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "ee3c3cc0553d46dcbc8aac28fcee5601"
format_stamp: "Formatted at 2026-08-07 22:39:49 on dist-test-slave-7kzw"
server_key: "db2551a1879ce4436c22df66028d6d20"
server_key_iv: "e3c0642fd8f1c3ee65d0b71a6d6d6270"
server_key_version: "encryptionkey@0"
I20260807 22:39:49.435598 27195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:49.436357 27216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:49.436538 27195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 22:39:49.436640 27195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "ee3c3cc0553d46dcbc8aac28fcee5601"
format_stamp: "Formatted at 2026-08-07 22:39:49 on dist-test-slave-7kzw"
server_key: "db2551a1879ce4436c22df66028d6d20"
server_key_iv: "e3c0642fd8f1c3ee65d0b71a6d6d6270"
server_key_version: "encryptionkey@0"
I20260807 22:39:49.436754 27195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:49.440094 27195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:49.441896 27195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:49.442056 27195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:49.446297 27195 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41361
I20260807 22:39:49.446332 27294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41361 every 8 connection(s)
I20260807 22:39:49.446687 27195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:49.447674 27295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:49.448323 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 27195
I20260807 22:39:49.448460 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:49.448644 29003 external_mini_cluster.cc:1428] Setting key f10f7b8badb6ce694608f54c28a7470a
I20260807 22:39:49.450877 27295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601: Bootstrap starting.
I20260807 22:39:49.451045 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:49.449176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:52891 (local address 127.0.0.1:41361)
0807 22:39:49.449390 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:49.449398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:49.449421 (+ 23us) server_negotiation.cc:407] Connection header received
0807 22:39:49.449810 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:49.449816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:49.449840 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:49.449945 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:49.450584 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:49.450587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:49.450588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:49.450591 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:49.450627 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:49.450636 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:49.450691 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:49.450792 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 22:39:49.450824 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260807 22:39:49.451719 27295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:49.452039 27295 log.cc:824] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:49.452152 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:49.452732 27295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601: No bootstrap required, opened a new log
I20260807 22:39:49.454964 27295 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3c3cc0553d46dcbc8aac28fcee5601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41361 } }
I20260807 22:39:49.455142 27295 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:49.455192 27295 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3c3cc0553d46dcbc8aac28fcee5601, State: Initialized, Role: FOLLOWER
I20260807 22:39:49.455338 27295 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3c3cc0553d46dcbc8aac28fcee5601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41361 } }
I20260807 22:39:49.455436 27295 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:49.455461 27295 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:49.455533 27295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:49.455677 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:49.453538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:43883 (local address 127.0.0.1:41361)
0807 22:39:49.453570 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:49.453572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:49.454651 (+ 1079us) server_negotiation.cc:407] Connection header received
0807 22:39:49.454715 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:49.454716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:49.454721 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:49.454791 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:49.455395 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:49.455400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:49.455401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:49.455405 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:49.455415 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:49.455419 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:49.455478 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:49.455601 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 22:39:49.455613 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 22:39:49.456281 27295 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3c3cc0553d46dcbc8aac28fcee5601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41361 } }
I20260807 22:39:49.456373 27295 leader_election.cc:302] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee3c3cc0553d46dcbc8aac28fcee5601; no voters:
I20260807 22:39:49.456489 27295 leader_election.cc:288] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:49.456545 27310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:49.456708 27310 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [term 1 LEADER]: Becoming Leader. State: Replica: ee3c3cc0553d46dcbc8aac28fcee5601, State: Running, Role: LEADER
I20260807 22:39:49.456746 27295 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:49.456856 27310 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3c3cc0553d46dcbc8aac28fcee5601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41361 } }
I20260807 22:39:49.457216 27312 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee3c3cc0553d46dcbc8aac28fcee5601. Latest consensus state: current_term: 1 leader_uuid: "ee3c3cc0553d46dcbc8aac28fcee5601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3c3cc0553d46dcbc8aac28fcee5601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41361 } } }
I20260807 22:39:49.457326 27312 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:49.457232 27311 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee3c3cc0553d46dcbc8aac28fcee5601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3c3cc0553d46dcbc8aac28fcee5601" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41361 } } }
I20260807 22:39:49.457394 27311 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:49.457767 27317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:49.458235 27317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:49.459890 27317 catalog_manager.cc:1347] Generated new cluster ID: ea840351a0094a2f9369c875b804c7f4
I20260807 22:39:49.459945 27317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:49.477448 27317 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:49.477648 27317 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:49.482708 27317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee3c3cc0553d46dcbc8aac28fcee5601: Generated new TSK 0
I20260807 22:39:49.482897 27317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:49.487946 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 27195
2026-08-07T22:39:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:49 dist-test-slave-7kzw krb5kdc[27343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:49 dist-test-slave-7kzw krb5kdc[27343](info): set up 2 sockets
Aug 07 22:39:49 dist-test-slave-7kzw krb5kdc[27343](info): commencing operation
krb5kdc: starting...
W20260807 22:39:50.712559 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:50 dist-test-slave-7kzw krb5kdc[27343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:50.833174 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34463
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:50.922669 27360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:50.922864 27360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:50.922887 27360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:50.923949 27360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:50.923983 27360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:50.923997 27360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:50.924010 27360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:50.924022 27360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:50.926076 27360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:50.926270 27360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:50.926436 27360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:50.931310 27371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:50.931294 27375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:50.931348 27372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:50.931555 27360 server_base.cc:1034] running on GCE node
I20260807 22:39:50.931808 27360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:50.932103 27360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:50.933279 27360 hybrid_clock.cc:648] HybridClock initialized: now 1786142390933256 us; error 45 us; skew 500 ppm
Aug 07 22:39:50 dist-test-slave-7kzw krb5kdc[27343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142390, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:50.935915 27360 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:50.936380 27360 webserver.cc:460] Webserver started at http://127.0.0.1:42149/ using document root <none> and password file <none>
I20260807 22:39:50.936626 27360 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:50.936702 27360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:50.936851 27360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:50.937878 27360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "a98bd876d6aa412cbab5c11682de33d3"
format_stamp: "Formatted at 2026-08-07 22:39:50 on dist-test-slave-7kzw"
server_key: "66b15d0458918fce7724d080ebc08e59"
server_key_iv: "efd033f91a7f8074fedca5328184f8b2"
server_key_version: "encryptionkey@0"
I20260807 22:39:50.938251 27360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "a98bd876d6aa412cbab5c11682de33d3"
format_stamp: "Formatted at 2026-08-07 22:39:50 on dist-test-slave-7kzw"
server_key: "66b15d0458918fce7724d080ebc08e59"
server_key_iv: "efd033f91a7f8074fedca5328184f8b2"
server_key_version: "encryptionkey@0"
I20260807 22:39:50.939633 27360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:50.940384 27389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:50.940563 27360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:50.940658 27360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "a98bd876d6aa412cbab5c11682de33d3"
format_stamp: "Formatted at 2026-08-07 22:39:50 on dist-test-slave-7kzw"
server_key: "66b15d0458918fce7724d080ebc08e59"
server_key_iv: "efd033f91a7f8074fedca5328184f8b2"
server_key_version: "encryptionkey@0"
I20260807 22:39:50.940742 27360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:50.950906 27360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:50.953122 27360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:50.953280 27360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:50.957340 27360 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34463
I20260807 22:39:50.957372 27464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34463 every 8 connection(s)
I20260807 22:39:50.957720 27360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:50.958626 27465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:50.960932 27465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3: Bootstrap starting.
I20260807 22:39:50.961550 27465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:50.961851 27465 log.cc:824] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:50.962633 27465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3: No bootstrap required, opened a new log
I20260807 22:39:50.964210 27465 raft_consensus.cc:348] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a98bd876d6aa412cbab5c11682de33d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34463 } }
I20260807 22:39:50.964362 27465 raft_consensus.cc:374] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:50.964386 27465 raft_consensus.cc:729] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a98bd876d6aa412cbab5c11682de33d3, State: Initialized, Role: FOLLOWER
I20260807 22:39:50.964550 27465 consensus_queue.cc:260] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a98bd876d6aa412cbab5c11682de33d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34463 } }
I20260807 22:39:50.964613 27465 raft_consensus.cc:388] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:50.964634 27465 raft_consensus.cc:482] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:50.964684 27465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:50.965227 27465 raft_consensus.cc:504] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a98bd876d6aa412cbab5c11682de33d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34463 } }
I20260807 22:39:50.965350 27465 leader_election.cc:302] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a98bd876d6aa412cbab5c11682de33d3; no voters:
I20260807 22:39:50.965531 27465 leader_election.cc:288] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:50.965575 27469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:50.965832 27469 raft_consensus.cc:686] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [term 1 LEADER]: Becoming Leader. State: Replica: a98bd876d6aa412cbab5c11682de33d3, State: Running, Role: LEADER
I20260807 22:39:50.965920 27465 sys_catalog.cc:564] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:50.965917 27469 consensus_queue.cc:237] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a98bd876d6aa412cbab5c11682de33d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34463 } }
I20260807 22:39:50.966358 27473 sys_catalog.cc:455] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a98bd876d6aa412cbab5c11682de33d3. Latest consensus state: current_term: 1 leader_uuid: "a98bd876d6aa412cbab5c11682de33d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a98bd876d6aa412cbab5c11682de33d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34463 } } }
I20260807 22:39:50.966452 27473 sys_catalog.cc:458] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:50.966609 27471 sys_catalog.cc:455] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a98bd876d6aa412cbab5c11682de33d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a98bd876d6aa412cbab5c11682de33d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34463 } } }
I20260807 22:39:50.966753 27471 sys_catalog.cc:458] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:50.966902 27480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:50.967618 27480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:50.967906 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 27360
I20260807 22:39:50.968055 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:50.968290 29003 external_mini_cluster.cc:1428] Setting key 4c9b772e72bba5e45d0efaaac1eaa473
I20260807 22:39:50.969409 27480 catalog_manager.cc:1347] Generated new cluster ID: b3e4f4a0e44d40d7963b5d3942b2c475
I20260807 22:39:50.969465 27480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:39:50 dist-test-slave-7kzw krb5kdc[27343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 22:39:50.990075 27499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:50.968699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:45703 (local address 127.0.0.1:34463)
0807 22:39:50.968868 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:50.968878 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:50.978714 (+ 9836us) server_negotiation.cc:407] Connection header received
0807 22:39:50.978883 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:50.978891 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:50.978950 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:50.979091 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:50.979833 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:50.981303 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:50.982325 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:50.982980 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:50.983092 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:50.986113 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:50.986117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:50.986117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:50.986131 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:50.988184 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:50.988653 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:50.988656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:50.988656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:50.988759 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:50.989008 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:50.989009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:50.989010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:50.989296 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:50.989462 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:50.989576 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 22:39:50.989669 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260807 22:39:50.991212 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:50.992756 27480 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:50.993325 27480 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:51.000033 27499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:50.992729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:38345 (local address 127.0.0.1:34463)
0807 22:39:50.992763 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:50.992765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:50.992899 (+ 134us) server_negotiation.cc:407] Connection header received
0807 22:39:50.992957 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:50.992958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:50.992970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:50.993028 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:50.993661 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:50.994382 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:50.995537 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:50.995835 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:50.995909 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:50.997267 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:50.997268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:50.997269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:50.997276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:50.998819 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:50.999243 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:50.999244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:50.999245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:50.999302 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:50.999533 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:50.999534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:50.999534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:50.999646 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:50.999760 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:50.999887 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 22:39:50.999947 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 22:39:51.002728 27480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a98bd876d6aa412cbab5c11682de33d3: Generated new TSK 0
I20260807 22:39:51.002913 27480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:51.004534 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 27360
W20260807 22:39:51.017642 27341 connection.cc:535] client connection to 127.0.0.1:34463 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34463): Connection reset by peer (error 104)
2026-08-07T22:39:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:51 dist-test-slave-7kzw krb5kdc[27512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:51 dist-test-slave-7kzw krb5kdc[27512](info): set up 2 sockets
Aug 07 22:39:51 dist-test-slave-7kzw krb5kdc[27512](info): commencing operation
krb5kdc: starting...
W20260807 22:39:52.215834 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:52 dist-test-slave-7kzw krb5kdc[27512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:52.339320 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:52.421497 27531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:52.421693 27531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:52.421716 27531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:52.422905 27531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:52.423002 27531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:52.423024 27531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:52.423048 27531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:52.423076 27531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:52.424643 27531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:52.424897 27531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:52.425127 27531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:52.429996 27536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:52.429987 27537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:52.430013 27539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:52.430231 27531 server_base.cc:1034] running on GCE node
I20260807 22:39:52.430595 27531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:52.430912 27531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:52.432055 27531 hybrid_clock.cc:648] HybridClock initialized: now 1786142392432020 us; error 46 us; skew 500 ppm
Aug 07 22:39:52 dist-test-slave-7kzw krb5kdc[27512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:52.434720 27531 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:52.435169 27531 webserver.cc:460] Webserver started at http://127.0.0.1:37911/ using document root <none> and password file <none>
I20260807 22:39:52.435379 27531 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:52.435420 27531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:52.435508 27531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:52.436493 27531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "9be54250dac1497387f4aea8739b32aa"
format_stamp: "Formatted at 2026-08-07 22:39:52 on dist-test-slave-7kzw"
server_key: "812d67001359c0a77e6dc915cf31296f"
server_key_iv: "32da5f91f2599f2e5d19f73c1571c018"
server_key_version: "encryptionkey@0"
I20260807 22:39:52.436798 27531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "9be54250dac1497387f4aea8739b32aa"
format_stamp: "Formatted at 2026-08-07 22:39:52 on dist-test-slave-7kzw"
server_key: "812d67001359c0a77e6dc915cf31296f"
server_key_iv: "32da5f91f2599f2e5d19f73c1571c018"
server_key_version: "encryptionkey@0"
I20260807 22:39:52.438098 27531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:39:52.438910 27548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:52.439105 27531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:39:52.439218 27531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "9be54250dac1497387f4aea8739b32aa"
format_stamp: "Formatted at 2026-08-07 22:39:52 on dist-test-slave-7kzw"
server_key: "812d67001359c0a77e6dc915cf31296f"
server_key_iv: "32da5f91f2599f2e5d19f73c1571c018"
server_key_version: "encryptionkey@0"
I20260807 22:39:52.439311 27531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:52.446094 27531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:52.448426 27531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:52.448585 27531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:52.452893 27531 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37335
I20260807 22:39:52.452934 27620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37335 every 8 connection(s)
I20260807 22:39:52.453287 27531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:52.453888 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 27531
I20260807 22:39:52.454030 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:52.454208 27621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:52.454233 29003 external_mini_cluster.cc:1428] Setting key ab074d2a3973ea8d5447e33fe51b0345
I20260807 22:39:52.457508 27621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa: Bootstrap starting.
I20260807 22:39:52.458140 27621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:52.458508 27621 log.cc:824] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:52.459244 27621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa: No bootstrap required, opened a new log
I20260807 22:39:52.459823 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:52.454799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:55335 (local address 127.0.0.1:37335)
0807 22:39:52.454981 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:52.454990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:52.455024 (+ 34us) server_negotiation.cc:407] Connection header received
0807 22:39:52.455391 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:52.455397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:52.455444 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:52.455554 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:52.455825 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:52.456704 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:52.457800 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:52.458188 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:52.458325 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:52.459112 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:52.459116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:52.459116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:52.459305 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:52.459346 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:52.459362 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:52.459443 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:52.459562 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 22:39:52.459611 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260807 22:39:52.460942 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:52.461676 27621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be54250dac1497387f4aea8739b32aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260807 22:39:52.461886 27621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:52.461930 27621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be54250dac1497387f4aea8739b32aa, State: Initialized, Role: FOLLOWER
I20260807 22:39:52.462067 27621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be54250dac1497387f4aea8739b32aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260807 22:39:52.462170 27621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:52.462208 27621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:52.462266 27621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:52.462935 27621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be54250dac1497387f4aea8739b32aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260807 22:39:52.463056 27621 leader_election.cc:302] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9be54250dac1497387f4aea8739b32aa; no voters:
I20260807 22:39:52.463253 27621 leader_election.cc:288] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:52.463318 27635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:52.463450 27635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [term 1 LEADER]: Becoming Leader. State: Replica: 9be54250dac1497387f4aea8739b32aa, State: Running, Role: LEADER
I20260807 22:39:52.463550 27621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:52.463541 27635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be54250dac1497387f4aea8739b32aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } }
I20260807 22:39:52.463910 27635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9be54250dac1497387f4aea8739b32aa. Latest consensus state: current_term: 1 leader_uuid: "9be54250dac1497387f4aea8739b32aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be54250dac1497387f4aea8739b32aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } } }
I20260807 22:39:52.463987 27635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:52.464151 27637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9be54250dac1497387f4aea8739b32aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be54250dac1497387f4aea8739b32aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37335 } } }
I20260807 22:39:52.464211 27637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:52.464314 27641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:52.464775 27641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:52.466051 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:52.462549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:41495 (local address 127.0.0.1:37335)
0807 22:39:52.462596 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:52.462598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:52.462664 (+ 66us) server_negotiation.cc:407] Connection header received
0807 22:39:52.462726 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:52.462728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:52.462737 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:52.462787 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:52.462928 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:52.463655 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:52.464701 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:52.465010 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:52.465078 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:52.465782 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:52.465783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:52.465784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:52.465787 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:52.465801 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:52.465805 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:52.465891 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:52.465965 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 22:39:52.465974 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:39:52.466732 27641 catalog_manager.cc:1347] Generated new cluster ID: 885c65c791ec48fdbda7c2dfe4e9935d
I20260807 22:39:52.466807 27641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:52.475611 27641 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:52.476136 27641 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:52.478758 27641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9be54250dac1497387f4aea8739b32aa: Generated new TSK 0
I20260807 22:39:52.478936 27641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:52.480834 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 27531
W20260807 22:39:52.491935 27510 connection.cc:535] client connection to 127.0.0.1:37335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37335): Connection reset by peer (error 104)
2026-08-07T22:39:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1473 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:52 dist-test-slave-7kzw krb5kdc[27664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:52 dist-test-slave-7kzw krb5kdc[27664](info): set up 2 sockets
Aug 07 22:39:52 dist-test-slave-7kzw krb5kdc[27664](info): commencing operation
krb5kdc: starting...
W20260807 22:39:53.701376 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:53 dist-test-slave-7kzw krb5kdc[27664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 22:39:53.824995 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41097
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:53.908960 27681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:53.909168 27681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:53.909190 27681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:53.910416 27681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:53.910478 27681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:53.910494 27681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:53.910507 27681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:53.910519 27681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:53.912001 27681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41097
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:53.912264 27681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:53.912498 27681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:53.917429 27687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:53.917449 27688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:53.917441 27691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:53.917769 27681 server_base.cc:1034] running on GCE node
I20260807 22:39:53.917986 27681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:53.918288 27681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:53.919479 27681 hybrid_clock.cc:648] HybridClock initialized: now 1786142393919457 us; error 56 us; skew 500 ppm
Aug 07 22:39:53 dist-test-slave-7kzw krb5kdc[27664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:53.922024 27681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 22:39:53.922430 27681 webserver.cc:460] Webserver started at http://127.0.0.1:41565/ using document root <none> and password file <none>
I20260807 22:39:53.922691 27681 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:53.922739 27681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:53.922842 27681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:53.923810 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "0acaad63b1a04b37b6c510dd51b89dd9"
format_stamp: "Formatted at 2026-08-07 22:39:53 on dist-test-slave-7kzw"
server_key: "7f56a91f0d07c03c761192c44ee96c0b"
server_key_iv: "8e887f43e214601f6f4f49aade85ead5"
server_key_version: "encryptionkey@0"
I20260807 22:39:53.924119 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "0acaad63b1a04b37b6c510dd51b89dd9"
format_stamp: "Formatted at 2026-08-07 22:39:53 on dist-test-slave-7kzw"
server_key: "7f56a91f0d07c03c761192c44ee96c0b"
server_key_iv: "8e887f43e214601f6f4f49aade85ead5"
server_key_version: "encryptionkey@0"
I20260807 22:39:53.925369 27681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 22:39:53.926077 27700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:53.926263 27681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:39:53.926354 27681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "0acaad63b1a04b37b6c510dd51b89dd9"
format_stamp: "Formatted at 2026-08-07 22:39:53 on dist-test-slave-7kzw"
server_key: "7f56a91f0d07c03c761192c44ee96c0b"
server_key_iv: "8e887f43e214601f6f4f49aade85ead5"
server_key_version: "encryptionkey@0"
I20260807 22:39:53.926503 27681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:53.929683 27681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:53.932047 27681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:53.932206 27681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:53.936088 27681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41097
I20260807 22:39:53.936147 27774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41097 every 8 connection(s)
I20260807 22:39:53.936484 27681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:53.937531 27775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:53.939813 27775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9: Bootstrap starting.
I20260807 22:39:53.940152 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 27681
I20260807 22:39:53.940296 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:53.940495 29003 external_mini_cluster.cc:1428] Setting key 557c8335272dea165c3bb8ee64c34621
I20260807 22:39:53.940495 27775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:53.940904 27775 log.cc:824] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:53.941758 27775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9: No bootstrap required, opened a new log
I20260807 22:39:53.943166 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:53.941022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:60177 (local address 127.0.0.1:41097)
0807 22:39:53.941223 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:53.941230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:53.941255 (+ 25us) server_negotiation.cc:407] Connection header received
0807 22:39:53.941850 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:53.941859 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:53.941898 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:53.942043 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:53.942722 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:53.942725 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:53.942725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:53.942730 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:53.942781 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:53.942796 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:53.942889 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:53.942926 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 22:39:53.942967 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260807 22:39:53.944311 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:53.944470 27775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0acaad63b1a04b37b6c510dd51b89dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41097 } }
I20260807 22:39:53.944628 27775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:53.944657 27775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0acaad63b1a04b37b6c510dd51b89dd9, State: Initialized, Role: FOLLOWER
I20260807 22:39:53.944773 27775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0acaad63b1a04b37b6c510dd51b89dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41097 } }
I20260807 22:39:53.944870 27775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:53.944904 27775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:53.944952 27775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:53.945737 27775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0acaad63b1a04b37b6c510dd51b89dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41097 } }
I20260807 22:39:53.945875 27775 leader_election.cc:302] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0acaad63b1a04b37b6c510dd51b89dd9; no voters:
I20260807 22:39:53.946034 27775 leader_election.cc:288] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:53.946141 27787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:53.946336 27775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:53.947248 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:53.946075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:48299 (local address 127.0.0.1:41097)
0807 22:39:53.946267 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:53.946269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:53.946278 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:39:53.946336 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:53.946337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:53.946342 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:53.946388 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:53.946989 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:53.946990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:53.946990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:53.946994 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:53.947010 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:53.947015 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:53.947073 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:53.947162 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 22:39:53.947177 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260807 22:39:53.946381 27787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [term 1 LEADER]: Becoming Leader. State: Replica: 0acaad63b1a04b37b6c510dd51b89dd9, State: Running, Role: LEADER
I20260807 22:39:53.947955 27787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0acaad63b1a04b37b6c510dd51b89dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41097 } }
I20260807 22:39:53.948648 27789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0acaad63b1a04b37b6c510dd51b89dd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0acaad63b1a04b37b6c510dd51b89dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41097 } } }
I20260807 22:39:53.948734 27791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0acaad63b1a04b37b6c510dd51b89dd9. Latest consensus state: current_term: 1 leader_uuid: "0acaad63b1a04b37b6c510dd51b89dd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0acaad63b1a04b37b6c510dd51b89dd9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41097 } } }
I20260807 22:39:53.948828 27789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:53.948840 27791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9 [sys.catalog]: This master's current role is: LEADER
W20260807 22:39:53.949046 27807 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:39:53.949105 27807 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:39:53.949168 27810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:53.949348 27810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:53.950794 27810 catalog_manager.cc:1347] Generated new cluster ID: 9a386a4d2427462996b98ad0b0ac5f15
I20260807 22:39:53.950850 27810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:53.965045 27810 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:53.965207 27810 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:53.972277 27810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0acaad63b1a04b37b6c510dd51b89dd9: Generated new TSK 0
I20260807 22:39:53.972465 27810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:53.978950 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 27681
2026-08-07T22:39:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:54 dist-test-slave-7kzw krb5kdc[27817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:54 dist-test-slave-7kzw krb5kdc[27817](info): set up 2 sockets
Aug 07 22:39:54 dist-test-slave-7kzw krb5kdc[27817](info): commencing operation
krb5kdc: starting...
W20260807 22:39:55.202718 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:55 dist-test-slave-7kzw krb5kdc[27817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:39:55.324708 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:41473
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:41473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:55.406549 27836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:55.406790 27836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:55.406827 27836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:55.407985 27836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:55.408056 27836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:55.408080 27836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:55.408107 27836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:55.408128 27836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:55.409575 27836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:41473
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:41473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:55.409854 27836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:55.410082 27836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:55.414891 27843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:55.414901 27849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:55.414915 27842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:55.415366 27836 server_base.cc:1034] running on GCE node
I20260807 22:39:55.415629 27836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:55.415864 27836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:55.417006 27836 hybrid_clock.cc:648] HybridClock initialized: now 1786142395416994 us; error 29 us; skew 500 ppm
Aug 07 22:39:55 dist-test-slave-7kzw krb5kdc[27817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142395, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:55.422829 27836 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:39:55.423296 27836 webserver.cc:460] Webserver started at http://127.28.82.254:40159/ using document root <none> and password file <none>
I20260807 22:39:55.423475 27836 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:55.423516 27836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:55.423602 27836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:55.424587 27836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "e6fb778bd7604dfaa3937a7e109ae0d8"
format_stamp: "Formatted at 2026-08-07 22:39:55 on dist-test-slave-7kzw"
server_key: "88e00e0451ee71e704efbc6466a58b4f"
server_key_iv: "0253740190f5eeea4e42bc233319826e"
server_key_version: "encryptionkey@0"
I20260807 22:39:55.424890 27836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "e6fb778bd7604dfaa3937a7e109ae0d8"
format_stamp: "Formatted at 2026-08-07 22:39:55 on dist-test-slave-7kzw"
server_key: "88e00e0451ee71e704efbc6466a58b4f"
server_key_iv: "0253740190f5eeea4e42bc233319826e"
server_key_version: "encryptionkey@0"
I20260807 22:39:55.426191 27836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:39:55.426975 27858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:55.427165 27836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:55.427287 27836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "e6fb778bd7604dfaa3937a7e109ae0d8"
format_stamp: "Formatted at 2026-08-07 22:39:55 on dist-test-slave-7kzw"
server_key: "88e00e0451ee71e704efbc6466a58b4f"
server_key_iv: "0253740190f5eeea4e42bc233319826e"
server_key_version: "encryptionkey@0"
I20260807 22:39:55.427363 27836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:55.444054 27836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:55.445906 27836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:55.446070 27836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:55.450402 27836 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:41473
I20260807 22:39:55.450455 27931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:41473 every 8 connection(s)
I20260807 22:39:55.450857 27836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:55.451700 27932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:55.454013 27932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8: Bootstrap starting.
I20260807 22:39:55.454620 27932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:55.454886 27932 log.cc:824] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:55.455556 27932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8: No bootstrap required, opened a new log
I20260807 22:39:55.457105 27932 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fb778bd7604dfaa3937a7e109ae0d8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41473 } }
I20260807 22:39:55.457239 27932 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:55.457262 27932 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6fb778bd7604dfaa3937a7e109ae0d8, State: Initialized, Role: FOLLOWER
I20260807 22:39:55.457360 27932 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fb778bd7604dfaa3937a7e109ae0d8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41473 } }
I20260807 22:39:55.457414 27932 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:55.457434 27932 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:55.457461 27932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:55.458272 27932 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fb778bd7604dfaa3937a7e109ae0d8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41473 } }
I20260807 22:39:55.458364 27932 leader_election.cc:302] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6fb778bd7604dfaa3937a7e109ae0d8; no voters:
I20260807 22:39:55.458541 27932 leader_election.cc:288] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:55.458613 27937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:55.458788 27937 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [term 1 LEADER]: Becoming Leader. State: Replica: e6fb778bd7604dfaa3937a7e109ae0d8, State: Running, Role: LEADER
I20260807 22:39:55.458839 27932 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:55.458904 27937 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fb778bd7604dfaa3937a7e109ae0d8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41473 } }
I20260807 22:39:55.459317 27938 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6fb778bd7604dfaa3937a7e109ae0d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fb778bd7604dfaa3937a7e109ae0d8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41473 } } }
I20260807 22:39:55.459424 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 27836
I20260807 22:39:55.459553 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:55.459595 27938 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:55.459791 29003 external_mini_cluster.cc:1428] Setting key a2ca242e7bc45bcd2ec5964e4c8fa165
I20260807 22:39:55.459514 27939 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6fb778bd7604dfaa3937a7e109ae0d8. Latest consensus state: current_term: 1 leader_uuid: "e6fb778bd7604dfaa3937a7e109ae0d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fb778bd7604dfaa3937a7e109ae0d8" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41473 } } }
I20260807 22:39:55.459918 27939 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:55.459981 27949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:55.460541 27949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:55.463132 27949 catalog_manager.cc:1347] Generated new cluster ID: 2c7166ec6f184ba2992122834f978d68
I20260807 22:39:55.463232 27949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:39:55 dist-test-slave-7kzw krb5kdc[27817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:55.471541 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:55.460444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49095 (local address 127.28.82.254:41473)
0807 22:39:55.460643 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:55.460652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:55.460687 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:39:55.460946 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:55.460954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:55.461010 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:55.461142 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:55.461934 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:55.462986 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:55.464073 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:55.464710 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:55.464836 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:55.467736 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:55.467746 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:55.467747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:55.467760 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:55.469749 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:55.470225 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:55.470227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:55.470228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:55.470352 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:55.470627 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:55.470628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:55.470628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:55.470934 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:55.471076 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:55.471229 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 22:39:55.471325 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
I20260807 22:39:55.472600 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:55.475630 27949 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:55.476120 27949 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:55.481669 27960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:55.474011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47929 (local address 127.28.82.254:41473)
0807 22:39:55.474044 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:55.474046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:55.474151 (+ 105us) server_negotiation.cc:407] Connection header received
0807 22:39:55.474196 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:55.474196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:55.474221 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:55.474281 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:55.474959 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:55.475553 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:55.476577 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:55.476903 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:55.476961 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:55.478377 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:55.478379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:55.478379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:55.478386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:55.480034 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:55.480507 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:55.480508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:55.480509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:55.480567 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:55.480828 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:55.480829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:55.480829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:55.480969 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:55.481066 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:55.481500 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 22:39:55.481563 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 22:39:55.484313 27949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6fb778bd7604dfaa3937a7e109ae0d8: Generated new TSK 0
I20260807 22:39:55.484503 27949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:55.487123 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 27836
W20260807 22:39:55.500535 27814 connection.cc:535] client connection to 127.28.82.254:41473 recv error: Network error: failed to read from TLS socket (remote: 127.28.82.254:41473): Connection reset by peer (error 104)
2026-08-07T22:39:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:55 dist-test-slave-7kzw krb5kdc[27977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:55 dist-test-slave-7kzw krb5kdc[27977](info): set up 2 sockets
Aug 07 22:39:55 dist-test-slave-7kzw krb5kdc[27977](info): commencing operation
krb5kdc: starting...
W20260807 22:39:56.710367 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:56 dist-test-slave-7kzw krb5kdc[27977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:39:56.833214 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43233
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:45799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:43233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:56.917111 27992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:56.917320 27992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:56.917342 27992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:56.918577 27992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:56.918632 27992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:56.918645 27992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:56.918658 27992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:56.918713 27992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:56.920156 27992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:45799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:43233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:43233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:56.920414 27992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:56.920665 27992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:56.925776 28003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:56.925805 28001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:56.925892 27992 server_base.cc:1034] running on GCE node
W20260807 22:39:56.925805 28006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:56.926267 27992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:56.926623 27992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:56.927809 27992 hybrid_clock.cc:648] HybridClock initialized: now 1786142396927802 us; error 38 us; skew 500 ppm
Aug 07 22:39:56 dist-test-slave-7kzw krb5kdc[27977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142396, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:56.930437 27992 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:39:56.930891 27992 webserver.cc:460] Webserver started at http://127.28.82.254:46083/ using document root <none> and password file <none>
I20260807 22:39:56.931114 27992 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:56.931159 27992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:56.931306 27992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:56.932348 27992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "0777efa92c9b4bcb84d2f619bc92b740"
format_stamp: "Formatted at 2026-08-07 22:39:56 on dist-test-slave-7kzw"
server_key: "1bca8f61afa7dbb993eee0802ecb27f6"
server_key_iv: "9afa28fb963a701862d95740fd9b7370"
server_key_version: "encryptionkey@0"
I20260807 22:39:56.932717 27992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "0777efa92c9b4bcb84d2f619bc92b740"
format_stamp: "Formatted at 2026-08-07 22:39:56 on dist-test-slave-7kzw"
server_key: "1bca8f61afa7dbb993eee0802ecb27f6"
server_key_iv: "9afa28fb963a701862d95740fd9b7370"
server_key_version: "encryptionkey@0"
I20260807 22:39:56.934180 27992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:56.935006 28018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:56.935189 27992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:56.935315 27992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "0777efa92c9b4bcb84d2f619bc92b740"
format_stamp: "Formatted at 2026-08-07 22:39:56 on dist-test-slave-7kzw"
server_key: "1bca8f61afa7dbb993eee0802ecb27f6"
server_key_iv: "9afa28fb963a701862d95740fd9b7370"
server_key_version: "encryptionkey@0"
I20260807 22:39:56.935395 27992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:56.944164 27992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:56.946528 27992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:56.946692 27992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:56.950860 27992 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:43233
I20260807 22:39:56.950892 28094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:43233 every 8 connection(s)
I20260807 22:39:56.951236 27992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:56.952093 28095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:56.954334 28095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740: Bootstrap starting.
I20260807 22:39:56.954941 28095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:56.955251 28095 log.cc:824] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:56.955963 28095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740: No bootstrap required, opened a new log
I20260807 22:39:56.957538 28095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777efa92c9b4bcb84d2f619bc92b740" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43233 } }
I20260807 22:39:56.957674 28095 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:56.957728 28095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0777efa92c9b4bcb84d2f619bc92b740, State: Initialized, Role: FOLLOWER
I20260807 22:39:56.957923 28095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777efa92c9b4bcb84d2f619bc92b740" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43233 } }
I20260807 22:39:56.958012 28095 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:56.958057 28095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:56.958108 28095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:56.958559 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 27992
I20260807 22:39:56.958705 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:56.958935 29003 external_mini_cluster.cc:1428] Setting key 31e0a54b858df193b9c4caaa04e10ddc
I20260807 22:39:56.959141 28095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777efa92c9b4bcb84d2f619bc92b740" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43233 } }
I20260807 22:39:56.959261 28095 leader_election.cc:302] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0777efa92c9b4bcb84d2f619bc92b740; no voters:
I20260807 22:39:56.959426 28095 leader_election.cc:288] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:56.959486 28098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:56.959656 28098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [term 1 LEADER]: Becoming Leader. State: Replica: 0777efa92c9b4bcb84d2f619bc92b740, State: Running, Role: LEADER
I20260807 22:39:56.959733 28095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:56.959793 28098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777efa92c9b4bcb84d2f619bc92b740" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43233 } }
I20260807 22:39:56.960111 28098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0777efa92c9b4bcb84d2f619bc92b740. Latest consensus state: current_term: 1 leader_uuid: "0777efa92c9b4bcb84d2f619bc92b740" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777efa92c9b4bcb84d2f619bc92b740" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43233 } } }
I20260807 22:39:56.960216 28098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:56.960129 28105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0777efa92c9b4bcb84d2f619bc92b740" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777efa92c9b4bcb84d2f619bc92b740" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 43233 } } }
I20260807 22:39:56.960384 28105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:56.960440 28112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:56.960980 28112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:56.962888 28112 catalog_manager.cc:1347] Generated new cluster ID: 72673d9f0cb645759149cfbc7581d7b9
I20260807 22:39:56.962941 28112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:56.965379 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:56.959396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37761 (local address 127.28.82.254:43233)
0807 22:39:56.959797 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:56.959807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:56.959840 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:39:56.959952 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:56.959958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:56.960011 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:56.960143 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:56.960511 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:56.962054 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:56.963184 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:56.963796 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:56.963912 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:56.964623 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:56.964627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:56.964627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:56.964837 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:56.964877 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:56.964892 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:56.964983 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:56.965068 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 22:39:56.965096 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":110,"threads_started":1}
I20260807 22:39:56.966638 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:56.971659 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:56.968236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56243 (local address 127.28.82.254:43233)
0807 22:39:56.968314 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:56.968316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:56.968324 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:39:56.968423 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:56.968424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:56.968434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:56.968480 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:56.968616 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:56.969340 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:56.970342 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:56.970671 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:56.970729 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:56.971402 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:56.971403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:56.971404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:56.971408 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:56.971423 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:56.971427 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:56.971484 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:56.971576 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 22:39:56.971588 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 22:39:56.973237 28112 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:56.973910 28112 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:56.980762 28112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0777efa92c9b4bcb84d2f619bc92b740: Generated new TSK 0
I20260807 22:39:56.980939 28112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:56.987249 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 27992
W20260807 22:39:56.998113 27974 connection.cc:535] client connection to 127.28.82.254:43233 recv error: Network error: failed to read from TLS socket (remote: 127.28.82.254:43233): Connection reset by peer (error 104)
2026-08-07T22:39:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:57 dist-test-slave-7kzw krb5kdc[28142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:57 dist-test-slave-7kzw krb5kdc[28142](info): set up 2 sockets
Aug 07 22:39:57 dist-test-slave-7kzw krb5kdc[28142](info): commencing operation
krb5kdc: starting...
W20260807 22:39:58.206315 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:58 dist-test-slave-7kzw krb5kdc[28142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:39:58.332855 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45139
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:40957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:45139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:58.415187 28165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:58.415390 28165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:58.415412 28165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:58.416482 28165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:58.416517 28165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:58.416531 28165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:58.416543 28165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:58.416555 28165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:58.417959 28165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:40957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:45139
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:58.418179 28165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:58.418391 28165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:58.423329 28175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:58.423373 28173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:58.423383 28172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:58.423489 28165 server_base.cc:1034] running on GCE node
I20260807 22:39:58.423802 28165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:58.424108 28165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:58.425284 28165 hybrid_clock.cc:648] HybridClock initialized: now 1786142398425267 us; error 41 us; skew 500 ppm
Aug 07 22:39:58 dist-test-slave-7kzw krb5kdc[28142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142398, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:58.427914 28165 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:39:58.428421 28165 webserver.cc:460] Webserver started at http://127.28.82.254:42875/ using document root <none> and password file <none>
I20260807 22:39:58.428670 28165 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:58.428742 28165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:58.428901 28165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:58.429943 28165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "b440d9a49f0e475eade4e05756942f76"
format_stamp: "Formatted at 2026-08-07 22:39:58 on dist-test-slave-7kzw"
server_key: "7f9a0f499626f287d766d3fd9cc67908"
server_key_iv: "d8b6977c0ccfce36fb8b9abfc216edc9"
server_key_version: "encryptionkey@0"
I20260807 22:39:58.430323 28165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "b440d9a49f0e475eade4e05756942f76"
format_stamp: "Formatted at 2026-08-07 22:39:58 on dist-test-slave-7kzw"
server_key: "7f9a0f499626f287d766d3fd9cc67908"
server_key_iv: "d8b6977c0ccfce36fb8b9abfc216edc9"
server_key_version: "encryptionkey@0"
I20260807 22:39:58.431795 28165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:58.432600 28183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:58.432781 28165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:39:58.432895 28165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "b440d9a49f0e475eade4e05756942f76"
format_stamp: "Formatted at 2026-08-07 22:39:58 on dist-test-slave-7kzw"
server_key: "7f9a0f499626f287d766d3fd9cc67908"
server_key_iv: "d8b6977c0ccfce36fb8b9abfc216edc9"
server_key_version: "encryptionkey@0"
I20260807 22:39:58.432973 28165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:58.436380 28165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:58.438727 28165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:58.438886 28165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:58.443279 28165 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:45139
I20260807 22:39:58.443331 28263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:45139 every 8 connection(s)
I20260807 22:39:58.443670 28165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:58.444655 28264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:58.447086 28264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76: Bootstrap starting.
I20260807 22:39:58.447695 28264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:58.447757 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 28165
I20260807 22:39:58.447885 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:58.448100 29003 external_mini_cluster.cc:1428] Setting key 55b02563bc0cd8adfd4cf9d7b6ec5322
I20260807 22:39:58.448091 28264 log.cc:824] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:58.448907 28264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76: No bootstrap required, opened a new log
I20260807 22:39:58.450843 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:58.448637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53245 (local address 127.28.82.254:45139)
0807 22:39:58.448857 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:58.448867 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:58.448902 (+ 35us) server_negotiation.cc:407] Connection header received
0807 22:39:58.449431 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:58.449440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:58.449475 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:58.449632 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:58.450337 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:58.450342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:58.450342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:58.450348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:58.450398 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:58.450413 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:58.450555 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:58.450593 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 22:39:58.450633 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":94,"threads_started":1}
I20260807 22:39:58.451789 28264 raft_consensus.cc:348] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b440d9a49f0e475eade4e05756942f76" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45139 } }
I20260807 22:39:58.451943 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:58.451973 28264 raft_consensus.cc:374] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:58.452008 28264 raft_consensus.cc:729] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b440d9a49f0e475eade4e05756942f76, State: Initialized, Role: FOLLOWER
I20260807 22:39:58.452118 28264 consensus_queue.cc:260] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b440d9a49f0e475eade4e05756942f76" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45139 } }
I20260807 22:39:58.452199 28264 raft_consensus.cc:388] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:58.452227 28264 raft_consensus.cc:482] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:58.452275 28264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:58.453027 28264 raft_consensus.cc:504] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b440d9a49f0e475eade4e05756942f76" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45139 } }
I20260807 22:39:58.453200 28264 leader_election.cc:302] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b440d9a49f0e475eade4e05756942f76; no voters:
I20260807 22:39:58.453389 28264 leader_election.cc:288] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:58.453413 28277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:58.453529 28277 raft_consensus.cc:686] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [term 1 LEADER]: Becoming Leader. State: Replica: b440d9a49f0e475eade4e05756942f76, State: Running, Role: LEADER
I20260807 22:39:58.453645 28277 consensus_queue.cc:237] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b440d9a49f0e475eade4e05756942f76" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45139 } }
I20260807 22:39:58.453694 28264 sys_catalog.cc:564] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:58.454061 28277 sys_catalog.cc:455] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b440d9a49f0e475eade4e05756942f76. Latest consensus state: current_term: 1 leader_uuid: "b440d9a49f0e475eade4e05756942f76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b440d9a49f0e475eade4e05756942f76" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45139 } } }
I20260807 22:39:58.454142 28277 sys_catalog.cc:458] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:58.454295 28278 sys_catalog.cc:455] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b440d9a49f0e475eade4e05756942f76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b440d9a49f0e475eade4e05756942f76" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45139 } } }
I20260807 22:39:58.454381 28278 sys_catalog.cc:458] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:58.454541 28284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:58.454916 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:58.453530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49791 (local address 127.28.82.254:45139)
0807 22:39:58.453623 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:58.453626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:58.453786 (+ 160us) server_negotiation.cc:407] Connection header received
0807 22:39:58.453879 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:58.453880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:58.453885 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:58.453911 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:58.454577 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:58.454578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:58.454578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:39:58.454583 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:58.454597 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:39:58.454602 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:58.454649 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:58.454842 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 22:39:58.454855 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:39:58.455076 28284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:58.456801 28284 catalog_manager.cc:1347] Generated new cluster ID: 0c4c77e1d9e54799ae8dc1bb544aa987
I20260807 22:39:58.456877 28284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:39:58.466297 28284 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:58.466526 28284 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:58.475631 28284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b440d9a49f0e475eade4e05756942f76: Generated new TSK 0
I20260807 22:39:58.475838 28284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:58.478143 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 28165
2026-08-07T22:39:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:39:58 dist-test-slave-7kzw krb5kdc[28304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:39:58 dist-test-slave-7kzw krb5kdc[28304](info): set up 2 sockets
Aug 07 22:39:58 dist-test-slave-7kzw krb5kdc[28304](info): commencing operation
krb5kdc: starting...
W20260807 22:39:59.699693 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:39:59 dist-test-slave-7kzw krb5kdc[28304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:39:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:39:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:39:59.820883 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:41215
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:39095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:41215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:39:59.905342 28327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:39:59.905541 28327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:39:59.905563 28327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:39:59.906747 28327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:39:59.906788 28327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:39:59.906802 28327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:39:59.906814 28327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:39:59.906864 28327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:39:59.908411 28327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:39095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:41215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:41215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:39:59.908654 28327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:39:59.908877 28327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:39:59.913887 28333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:59.913898 28334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:39:59.913899 28336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:39:59.914167 28327 server_base.cc:1034] running on GCE node
I20260807 22:39:59.914412 28327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:39:59.914732 28327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:39:59.915911 28327 hybrid_clock.cc:648] HybridClock initialized: now 1786142399915894 us; error 30 us; skew 500 ppm
Aug 07 22:39:59 dist-test-slave-7kzw krb5kdc[28304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142399, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:39:59.918254 28327 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:39:59.918756 28327 webserver.cc:460] Webserver started at http://127.28.82.254:41369/ using document root <none> and password file <none>
I20260807 22:39:59.918987 28327 fs_manager.cc:359] Metadata directory not provided
I20260807 22:39:59.919052 28327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:39:59.919200 28327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:39:59.920197 28327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "5fef9d4b0f64415588a75858dd1a3c44"
format_stamp: "Formatted at 2026-08-07 22:39:59 on dist-test-slave-7kzw"
server_key: "e2962a1c622c0f951111f7c718711a90"
server_key_iv: "2464a7379f8be86f561d6409d70087d6"
server_key_version: "encryptionkey@0"
I20260807 22:39:59.920547 28327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "5fef9d4b0f64415588a75858dd1a3c44"
format_stamp: "Formatted at 2026-08-07 22:39:59 on dist-test-slave-7kzw"
server_key: "e2962a1c622c0f951111f7c718711a90"
server_key_iv: "2464a7379f8be86f561d6409d70087d6"
server_key_version: "encryptionkey@0"
I20260807 22:39:59.921834 28327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:39:59.922555 28347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:39:59.922742 28327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:39:59.922843 28327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "5fef9d4b0f64415588a75858dd1a3c44"
format_stamp: "Formatted at 2026-08-07 22:39:59 on dist-test-slave-7kzw"
server_key: "e2962a1c622c0f951111f7c718711a90"
server_key_iv: "2464a7379f8be86f561d6409d70087d6"
server_key_version: "encryptionkey@0"
I20260807 22:39:59.922924 28327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:39:59.932374 28327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:39:59.934031 28327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:39:59.934186 28327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:39:59.938127 28327 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:41215
I20260807 22:39:59.938158 28426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:41215 every 8 connection(s)
I20260807 22:39:59.938498 28327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:39:59.939374 28427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:39:59.941701 28427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44: Bootstrap starting.
I20260807 22:39:59.942278 28427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44: Neither blocks nor log segments found. Creating new log.
I20260807 22:39:59.942636 28427 log.cc:824] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44: Log is configured to *not* fsync() on all Append() calls
I20260807 22:39:59.943320 28427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44: No bootstrap required, opened a new log
I20260807 22:39:59.944913 28427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fef9d4b0f64415588a75858dd1a3c44" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41215 } }
I20260807 22:39:59.945047 28427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:39:59.945101 28427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fef9d4b0f64415588a75858dd1a3c44, State: Initialized, Role: FOLLOWER
I20260807 22:39:59.945246 28427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fef9d4b0f64415588a75858dd1a3c44" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41215 } }
I20260807 22:39:59.945313 28427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:39:59.945350 28427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:39:59.945401 28427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:39:59.946240 28427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fef9d4b0f64415588a75858dd1a3c44" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41215 } }
I20260807 22:39:59.946288 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 28327
I20260807 22:39:59.946357 28427 leader_election.cc:302] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fef9d4b0f64415588a75858dd1a3c44; no voters:
I20260807 22:39:59.946449 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:39:59.946571 28427 leader_election.cc:288] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:39:59.946593 28432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:39:59.946714 29003 external_mini_cluster.cc:1428] Setting key c8bc0036480625bf3b3bdded325b30ba
I20260807 22:39:59.946732 28432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [term 1 LEADER]: Becoming Leader. State: Replica: 5fef9d4b0f64415588a75858dd1a3c44, State: Running, Role: LEADER
I20260807 22:39:59.946825 28432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fef9d4b0f64415588a75858dd1a3c44" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41215 } }
I20260807 22:39:59.946904 28427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:39:59.947139 28432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fef9d4b0f64415588a75858dd1a3c44. Latest consensus state: current_term: 1 leader_uuid: "5fef9d4b0f64415588a75858dd1a3c44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fef9d4b0f64415588a75858dd1a3c44" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41215 } } }
I20260807 22:39:59.947216 28432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:59.947153 28435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fef9d4b0f64415588a75858dd1a3c44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fef9d4b0f64415588a75858dd1a3c44" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 41215 } } }
I20260807 22:39:59.947257 28435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44 [sys.catalog]: This master's current role is: LEADER
I20260807 22:39:59.947649 28441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:39:59.948220 28441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:39:59.952656 28441 catalog_manager.cc:1347] Generated new cluster ID: 45e91e5f28dd4571aa34d4e72f9cd437
I20260807 22:39:59.952725 28441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 22:39:59 dist-test-slave-7kzw krb5kdc[28304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:39:59.957911 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:59.947558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34681 (local address 127.28.82.254:41215)
0807 22:39:59.948058 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:59.948067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:59.948106 (+ 39us) server_negotiation.cc:407] Connection header received
0807 22:39:59.948221 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:59.948228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:59.948297 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:59.948442 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:59.949111 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:59.950490 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:59.951493 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:59.952127 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:59.952239 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:59.954693 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:59.954696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:59.954697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:59.954708 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:59.956324 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:59.956711 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:59.956715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:59.956715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:59.956819 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:59.957031 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:59.957032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:59.957032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:59.957275 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:59.957441 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:59.957513 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 22:39:59.957750 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":76,"threads_started":1}
I20260807 22:39:59.958781 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:39:59.965960 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:39:59.960098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39547 (local address 127.28.82.254:41215)
0807 22:39:59.960136 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 22:39:59.960137 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:39:59.960260 (+ 123us) server_negotiation.cc:407] Connection header received
0807 22:39:59.960330 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:39:59.960331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:39:59.960343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:39:59.960398 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:39:59.960900 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:59.961360 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:59.962034 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:39:59.962250 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:39:59.962324 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:39:59.963462 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:39:59.963464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:39:59.963464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:39:59.963471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:39:59.964819 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:59.965178 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:59.965179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:59.965179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:59.965240 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:39:59.965443 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:39:59.965444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:39:59.965444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:39:59.965549 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:39:59.965644 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 22:39:59.965782 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 22:39:59.965855 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:39:59.979677 28441 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:39:59.980198 28441 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:39:59.989665 28441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fef9d4b0f64415588a75858dd1a3c44: Generated new TSK 0
I20260807 22:39:59.989857 28441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:39:59.998612 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 28327
2026-08-07T22:40:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:40:00 dist-test-slave-7kzw krb5kdc[28468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:40:00 dist-test-slave-7kzw krb5kdc[28468](info): set up 2 sockets
Aug 07 22:40:00 dist-test-slave-7kzw krb5kdc[28468](info): commencing operation
krb5kdc: starting...
W20260807 22:40:01.252781 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:40:01 dist-test-slave-7kzw krb5kdc[28468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:40:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:40:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:40:01.385519 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:33607
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:34621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:33607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:40:01.469115 28482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:40:01.469360 28482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:40:01.469385 28482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:40:01.470538 28482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:40:01.470577 28482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:40:01.470590 28482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:40:01.470603 28482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:40:01.470615 28482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:40:01.472187 28482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:34621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:33607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:33607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:40:01.472479 28482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:40:01.472723 28482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:40:01.477844 28494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:01.477953 28482 server_base.cc:1034] running on GCE node
W20260807 22:40:01.477829 28490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:01.477826 28491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:01.478389 28482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:40:01.478721 28482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:40:01.479884 28482 hybrid_clock.cc:648] HybridClock initialized: now 1786142401479869 us; error 30 us; skew 500 ppm
Aug 07 22:40:01 dist-test-slave-7kzw krb5kdc[28468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142401, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:40:01.487763 28482 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:40:01.488258 28482 webserver.cc:460] Webserver started at http://127.28.82.254:33545/ using document root <none> and password file <none>
I20260807 22:40:01.488508 28482 fs_manager.cc:359] Metadata directory not provided
I20260807 22:40:01.488577 28482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:40:01.488732 28482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:40:01.489861 28482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "f3de0ea6363a41f49a7e0545c915ac2e"
format_stamp: "Formatted at 2026-08-07 22:40:01 on dist-test-slave-7kzw"
server_key: "e0de476359993c64f29e195abfe60a97"
server_key_iv: "f19bb253d8af0588815eb53609c5f29c"
server_key_version: "encryptionkey@0"
I20260807 22:40:01.490247 28482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "f3de0ea6363a41f49a7e0545c915ac2e"
format_stamp: "Formatted at 2026-08-07 22:40:01 on dist-test-slave-7kzw"
server_key: "e0de476359993c64f29e195abfe60a97"
server_key_iv: "f19bb253d8af0588815eb53609c5f29c"
server_key_version: "encryptionkey@0"
I20260807 22:40:01.491787 28482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:40:01.492573 28505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:40:01.492879 28482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:40:01.492992 28482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "f3de0ea6363a41f49a7e0545c915ac2e"
format_stamp: "Formatted at 2026-08-07 22:40:01 on dist-test-slave-7kzw"
server_key: "e0de476359993c64f29e195abfe60a97"
server_key_iv: "f19bb253d8af0588815eb53609c5f29c"
server_key_version: "encryptionkey@0"
I20260807 22:40:01.493077 28482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:40:01.502393 28482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:40:01.504390 28482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:40:01.504559 28482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:40:01.508809 28482 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:33607
I20260807 22:40:01.508846 28585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:33607 every 8 connection(s)
I20260807 22:40:01.509217 28482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:40:01.510145 28586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:40:01.511380 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 28482
I20260807 22:40:01.511510 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:40:01.511710 29003 external_mini_cluster.cc:1428] Setting key caf46d4973b3164ed8b4337095cc20bd
I20260807 22:40:01.513082 28586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e: Bootstrap starting.
I20260807 22:40:01.513836 28586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e: Neither blocks nor log segments found. Creating new log.
I20260807 22:40:01.514175 28586 log.cc:824] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e: Log is configured to *not* fsync() on all Append() calls
I20260807 22:40:01.515043 28586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e: No bootstrap required, opened a new log
I20260807 22:40:01.517407 28586 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3de0ea6363a41f49a7e0545c915ac2e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33607 } }
I20260807 22:40:01.517604 28586 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:40:01.517647 28586 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3de0ea6363a41f49a7e0545c915ac2e, State: Initialized, Role: FOLLOWER
I20260807 22:40:01.517783 28586 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3de0ea6363a41f49a7e0545c915ac2e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33607 } }
I20260807 22:40:01.517879 28586 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:40:01.517973 28586 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:40:01.518041 28586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:40:01.518213 28589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:01.512294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45167 (local address 127.28.82.254:33607)
0807 22:40:01.512542 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:01.512552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:01.512590 (+ 38us) server_negotiation.cc:407] Connection header received
0807 22:40:01.513157 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:01.513166 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:01.513229 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:01.513383 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:01.513575 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:01.514967 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:01.515818 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:01.516459 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:01.516608 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:01.517323 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:01.517328 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:01.517328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:40:01.517558 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:01.517604 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:40:01.517620 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:01.517889 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:01.517922 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 22:40:01.517994 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":115,"threads_started":1}
I20260807 22:40:01.518776 28586 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3de0ea6363a41f49a7e0545c915ac2e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33607 } }
I20260807 22:40:01.518886 28586 leader_election.cc:302] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3de0ea6363a41f49a7e0545c915ac2e; no voters:
I20260807 22:40:01.519129 28586 leader_election.cc:288] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:40:01.519152 28592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:40:01.519284 28592 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [term 1 LEADER]: Becoming Leader. State: Replica: f3de0ea6363a41f49a7e0545c915ac2e, State: Running, Role: LEADER
I20260807 22:40:01.519361 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:40:01.519409 28586 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:40:01.520524 28592 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3de0ea6363a41f49a7e0545c915ac2e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33607 } }
I20260807 22:40:01.521098 28595 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3de0ea6363a41f49a7e0545c915ac2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3de0ea6363a41f49a7e0545c915ac2e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33607 } } }
I20260807 22:40:01.521195 28595 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [sys.catalog]: This master's current role is: LEADER
I20260807 22:40:01.521438 28616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:40:01.521487 28593 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3de0ea6363a41f49a7e0545c915ac2e. Latest consensus state: current_term: 1 leader_uuid: "f3de0ea6363a41f49a7e0545c915ac2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3de0ea6363a41f49a7e0545c915ac2e" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 33607 } } }
I20260807 22:40:01.521606 28593 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e [sys.catalog]: This master's current role is: LEADER
I20260807 22:40:01.522017 28616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:40:01.523717 28616 catalog_manager.cc:1347] Generated new cluster ID: 3da9e5ab513c44908ed3bb0c242455fc
I20260807 22:40:01.523775 28616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:40:01.524410 28589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:01.520843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60227 (local address 127.28.82.254:33607)
0807 22:40:01.520876 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:01.520878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:01.520888 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:40:01.520936 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:01.520937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:01.520947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:01.521010 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:01.521182 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:01.521955 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:01.523077 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:01.523322 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:01.523419 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:01.524158 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:01.524159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:01.524160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:40:01.524164 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:01.524178 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:40:01.524183 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:01.524231 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:01.524287 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 22:40:01.524309 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":16}
I20260807 22:40:01.529322 28616 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:40:01.530017 28616 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:40:01.533250 28616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3de0ea6363a41f49a7e0545c915ac2e: Generated new TSK 0
I20260807 22:40:01.533423 28616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:40:01.537045 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 28482
2026-08-07T22:40:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1535 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:40:01 dist-test-slave-7kzw krb5kdc[28633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:40:01 dist-test-slave-7kzw krb5kdc[28633](info): set up 2 sockets
Aug 07 22:40:01 dist-test-slave-7kzw krb5kdc[28633](info): commencing operation
krb5kdc: starting...
W20260807 22:40:02.787631 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:40:02 dist-test-slave-7kzw krb5kdc[28633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:40:02.917574 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:36713
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:36713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:40:03.000885 28649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:40:03.001096 28649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:40:03.001118 28649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:40:03.002198 28649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:40:03.002233 28649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:40:03.002247 28649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:40:03.002259 28649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:40:03.002271 28649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:40:03.003808 28649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:36713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:36713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:40:03.004065 28649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:40:03.004323 28649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:40:03.009341 28660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:03.009347 28657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:03.009379 28658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:03.009639 28649 server_base.cc:1034] running on GCE node
I20260807 22:40:03.010017 28649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:40:03.010317 28649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:40:03.011462 28649 hybrid_clock.cc:648] HybridClock initialized: now 1786142403011450 us; error 32 us; skew 500 ppm
Aug 07 22:40:03 dist-test-slave-7kzw krb5kdc[28633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142403, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:40:03.014083 28649 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:40:03.014618 28649 webserver.cc:460] Webserver started at http://127.28.82.254:44663/ using document root <none> and password file <none>
I20260807 22:40:03.014881 28649 fs_manager.cc:359] Metadata directory not provided
I20260807 22:40:03.014951 28649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:40:03.015105 28649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:40:03.016151 28649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "9b8e46cc76be4544bdb6265a696c41a7"
format_stamp: "Formatted at 2026-08-07 22:40:03 on dist-test-slave-7kzw"
server_key: "31311e7a67c49830ec6d09989bd8a46e"
server_key_iv: "907fe1aac2492d24c700335e2af39f3e"
server_key_version: "encryptionkey@0"
I20260807 22:40:03.016532 28649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "9b8e46cc76be4544bdb6265a696c41a7"
format_stamp: "Formatted at 2026-08-07 22:40:03 on dist-test-slave-7kzw"
server_key: "31311e7a67c49830ec6d09989bd8a46e"
server_key_iv: "907fe1aac2492d24c700335e2af39f3e"
server_key_version: "encryptionkey@0"
I20260807 22:40:03.017925 28649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:40:03.018806 28668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:40:03.018977 28649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:40:03.019084 28649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "9b8e46cc76be4544bdb6265a696c41a7"
format_stamp: "Formatted at 2026-08-07 22:40:03 on dist-test-slave-7kzw"
server_key: "31311e7a67c49830ec6d09989bd8a46e"
server_key_iv: "907fe1aac2492d24c700335e2af39f3e"
server_key_version: "encryptionkey@0"
I20260807 22:40:03.019158 28649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:40:03.022513 28649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:40:03.024859 28649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:40:03.025022 28649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:40:03.029477 28649 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:36713
I20260807 22:40:03.029532 28742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:36713 every 8 connection(s)
I20260807 22:40:03.029906 28649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:40:03.031104 28743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:40:03.032855 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 28649
I20260807 22:40:03.033001 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:40:03.033229 29003 external_mini_cluster.cc:1428] Setting key 1b1b34504deeb21ac64723b2b1f28e44
I20260807 22:40:03.033877 28743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7: Bootstrap starting.
I20260807 22:40:03.034581 28743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7: Neither blocks nor log segments found. Creating new log.
I20260807 22:40:03.034943 28743 log.cc:824] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7: Log is configured to *not* fsync() on all Append() calls
I20260807 22:40:03.035805 28743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7: No bootstrap required, opened a new log
I20260807 22:40:03.036209 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:03.033847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35491 (local address 127.28.82.254:36713)
0807 22:40:03.034094 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:03.034102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:03.034129 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:40:03.034629 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:03.034637 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:03.034680 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:03.034820 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:03.035756 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:03.035759 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:03.035759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:40:03.035763 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:03.035802 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:40:03.035812 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:03.035931 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:03.035957 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 22:40:03.036005 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
I20260807 22:40:03.037177 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:40:03.038125 28743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8e46cc76be4544bdb6265a696c41a7" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36713 } }
I20260807 22:40:03.038296 28743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 22:40:03.038352 28743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8e46cc76be4544bdb6265a696c41a7, State: Initialized, Role: FOLLOWER
I20260807 22:40:03.038515 28743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8e46cc76be4544bdb6265a696c41a7" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36713 } }
I20260807 22:40:03.038607 28743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 22:40:03.038647 28743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 22:40:03.038757 28743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:40:03.039392 28743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8e46cc76be4544bdb6265a696c41a7" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36713 } }
I20260807 22:40:03.039516 28743 leader_election.cc:302] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b8e46cc76be4544bdb6265a696c41a7; no voters:
I20260807 22:40:03.039710 28743 leader_election.cc:288] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 22:40:03.039844 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:03.038716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41167 (local address 127.28.82.254:36713)
0807 22:40:03.038752 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:03.038754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:03.038852 (+ 98us) server_negotiation.cc:407] Connection header received
0807 22:40:03.038921 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:03.038922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:03.038927 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:03.038993 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:03.039560 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:03.039561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:03.039561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 22:40:03.039566 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:03.039582 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 22:40:03.039586 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:03.039638 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:03.039748 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 22:40:03.039763 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:40:03.039731 28755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:40:03.040001 28743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:40:03.040035 28755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8e46cc76be4544bdb6265a696c41a7, State: Running, Role: LEADER
I20260807 22:40:03.040311 28755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8e46cc76be4544bdb6265a696c41a7" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36713 } }
I20260807 22:40:03.040670 28755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b8e46cc76be4544bdb6265a696c41a7. Latest consensus state: current_term: 1 leader_uuid: "9b8e46cc76be4544bdb6265a696c41a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8e46cc76be4544bdb6265a696c41a7" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36713 } } }
I20260807 22:40:03.040778 28757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b8e46cc76be4544bdb6265a696c41a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8e46cc76be4544bdb6265a696c41a7" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 36713 } } }
I20260807 22:40:03.040877 28757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:40:03.040789 28755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7 [sys.catalog]: This master's current role is: LEADER
I20260807 22:40:03.041081 28767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:40:03.041577 28767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:40:03.043200 28767 catalog_manager.cc:1347] Generated new cluster ID: 2cb6a5f6fbaf4507aaeb97c2acba4f74
I20260807 22:40:03.043270 28767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:40:03.049245 28767 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:40:03.049408 28767 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:40:03.053745 28767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b8e46cc76be4544bdb6265a696c41a7: Generated new TSK 0
I20260807 22:40:03.053925 28767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:40:03.061952 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 28649
2026-08-07T22:40:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1524 ms)
[----------] 18 tests from AuthTokenIssuingTest (27238 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:40:03 dist-test-slave-7kzw krb5kdc[28785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:40:03 dist-test-slave-7kzw krb5kdc[28785](info): set up 2 sockets
Aug 07 22:40:03 dist-test-slave-7kzw krb5kdc[28785](info): commencing operation
krb5kdc: starting...
W20260807 22:40:04.281560 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:40:04.411490 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39737
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39737,127.28.82.253:43145,127.28.82.252:41833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:40:04.495596 28806 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:40:04.495893 28806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:40:04.495927 28806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:40:04.495973 28806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:40:04.497148 28806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:40:04.497222 28806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:40:04.497246 28806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:40:04.497265 28806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:40:04.497295 28806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:40:04.498850 28806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39737,127.28.82.253:43145,127.28.82.252:41833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:40:04.499142 28806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:40:04.499385 28806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:40:04.504377 28814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:04.504397 28818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:04.504388 28815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:04.504698 28806 server_base.cc:1034] running on GCE node
I20260807 22:40:04.504966 28806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:40:04.505262 28806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:40:04.506402 28806 hybrid_clock.cc:648] HybridClock initialized: now 1786142404506387 us; error 25 us; skew 500 ppm
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:40:04.509052 28806 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:40:04.509490 28806 webserver.cc:460] Webserver started at http://127.28.82.254:44763/ using document root <none> and password file <none>
I20260807 22:40:04.509709 28806 fs_manager.cc:359] Metadata directory not provided
I20260807 22:40:04.509761 28806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:40:04.509927 28806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:40:04.511063 28806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "8a91c415ca9b4288a98af9cd44f345aa"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "27bb8f28c1ee8feba28894ff4920b76b"
server_key_iv: "f082f08f1c709b62e900d7c711b1e343"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.511427 28806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "8a91c415ca9b4288a98af9cd44f345aa"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "27bb8f28c1ee8feba28894ff4920b76b"
server_key_iv: "f082f08f1c709b62e900d7c711b1e343"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.512778 28806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 22:40:04.513494 28826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:40:04.513664 28806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:40:04.513788 28806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "8a91c415ca9b4288a98af9cd44f345aa"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "27bb8f28c1ee8feba28894ff4920b76b"
server_key_iv: "f082f08f1c709b62e900d7c711b1e343"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.513864 28806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:40:04.533390 28806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:40:04.535822 28806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:40:04.535996 28806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:40:04.540414 28806 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:39737
I20260807 22:40:04.540458 28895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:39737 every 8 connection(s)
I20260807 22:40:04.540866 28806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:40:04.541961 28896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:40:04.543684 28896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:04.546975 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 28806
I20260807 22:40:04.547125 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:40:04.547353 29003 external_mini_cluster.cc:1428] Setting key 0d91a502ebc4a5c188a2bed5630a9d41
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:40:04.555889 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.544461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39737 (local address 127.0.0.1:56052)
0807 22:40:04.544731 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.544816 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.545325 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.545612 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.545615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.545912 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.546072 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.546076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.547196 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.547198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.548320 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.548325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.548897 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.548899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.548985 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.549690 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.549700 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.551477 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.553903 (+ 2426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.553910 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.553912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.554141 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.554526 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.554528 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.554532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.554637 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.555206 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.555214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.555279 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.555374 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.555711 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":134,"thread_start_us":104,"threads_started":1}
I20260807 22:40:04.555897 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.544669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.28.82.254:39737)
0807 22:40:04.544867 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.544872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.544905 (+ 33us) server_negotiation.cc:407] Connection header received
0807 22:40:04.545491 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.545493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.545510 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.545613 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.546195 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.547087 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.548430 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.548779 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.548872 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.551571 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.551581 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.551582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.551590 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.553805 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.554232 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.554235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.554236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.554372 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.554713 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.554714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.554715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.555055 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.555157 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.555439 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.555594 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260807 22:40:04.556782 28896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:04.557145 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.557027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38140)
0807 22:40:04.557076 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.557125 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 22:40:04.557293 28831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.28.82.253@KRBTEST.COM; defaulting to no policy
I20260807 22:40:04.557785 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.557663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38154)
0807 22:40:04.557724 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.557754 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 22:40:04.557921 28896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:43145: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab.
Entry for principal kudu/127.28.82.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab.
I20260807 22:40:04.578722 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.253:43145
--webserver_interface=127.28.82.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab
--principal=kudu/127.28.82.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39737,127.28.82.253:43145,127.28.82.252:41833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:40:04.607152 28896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } attempt: 1
I20260807 22:40:04.607795 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.607637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38162)
0807 22:40:04.607711 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.607759 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 22:40:04.608217 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.608127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38178)
0807 22:40:04.608170 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.608200 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:40:04.608337 28896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:43145: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
W20260807 22:40:04.664376 28906 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:40:04.664649 28906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:40:04.664690 28906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:40:04.664733 28906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:40:04.665885 28906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:40:04.665951 28906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:40:04.665974 28906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:40:04.666007 28906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:40:04.666033 28906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:40:04.667618 28906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39737,127.28.82.253:43145,127.28.82.252:41833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab
--principal=kudu/127.28.82.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.253:43145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.82.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:40:04.667897 28906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:40:04.668133 28906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:40:04.673159 28915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:04.673142 28912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:04.673125 28919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:04.673202 28906 server_base.cc:1034] running on GCE node
I20260807 22:40:04.673688 28906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:40:04.674008 28906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:40:04.675179 28906 hybrid_clock.cc:648] HybridClock initialized: now 1786142404675159 us; error 47 us; skew 500 ppm
I20260807 22:40:04.676498 28896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } attempt: 2
I20260807 22:40:04.677049 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.676917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38194)
0807 22:40:04.676982 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.677020 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 22:40:04.677426 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.677332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38198)
0807 22:40:04.677375 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.677405 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 22:40:04.677516 28896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:43145: Network error: Client connection negotiation failed: client connection to 127.28.82.253:43145: connect: Connection refused (error 111)
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:40:04.678207 28906 init.cc:377] Logged in from keytab as kudu/127.28.82.253@KRBTEST.COM (short username kudu)
I20260807 22:40:04.678753 28906 webserver.cc:460] Webserver started at http://127.28.82.253:43855/ using document root <none> and password file <none>
I20260807 22:40:04.678988 28906 fs_manager.cc:359] Metadata directory not provided
I20260807 22:40:04.679056 28906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:40:04.679245 28906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:40:04.680371 28906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/data/instance:
uuid: "db521181b01245229ff3f84fa58aba4d"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "606791770f46b5aec12c168581349784"
server_key_iv: "de8b92446e888434500e6f7e1554bfc5"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.680760 28906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/wal/instance:
uuid: "db521181b01245229ff3f84fa58aba4d"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "606791770f46b5aec12c168581349784"
server_key_iv: "de8b92446e888434500e6f7e1554bfc5"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.682195 28906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:40:04.683097 28928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:40:04.683435 28906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 22:40:04.683561 28906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/wal
uuid: "db521181b01245229ff3f84fa58aba4d"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "606791770f46b5aec12c168581349784"
server_key_iv: "de8b92446e888434500e6f7e1554bfc5"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.683640 28906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:40:04.701869 28906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:40:04.704011 28906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:40:04.704180 28906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:40:04.708658 28906 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.253:43145
I20260807 22:40:04.708702 28999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.253:43145 every 8 connection(s)
I20260807 22:40:04.709086 28906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
I20260807 22:40:04.710094 29000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:40:04.711782 29000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:04.714821 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 28906
I20260807 22:40:04.714924 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-1/wal/instance
I20260807 22:40:04.715112 29003 external_mini_cluster.cc:1428] Setting key 4a4dbb5d256c9f84eb063cafab1ebdae
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:40:04.722016 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.712525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.28.82.254:39737)
0807 22:40:04.712592 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.712595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.712863 (+ 268us) server_negotiation.cc:407] Connection header received
0807 22:40:04.713334 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.713335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.713349 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.713410 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.714032 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.714644 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.715794 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.716018 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.716097 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.718809 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.718811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.718811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.718819 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.720367 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.720956 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.720957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.720958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.721017 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.721345 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.721346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.721346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.721456 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.721505 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.721877 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.721949 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 22:40:04.722357 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.712514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39737 (local address 127.0.0.1:56058)
0807 22:40:04.712728 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.712765 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.713244 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.713481 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.713484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.713755 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.713953 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.713957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.714743 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.714744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.715697 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.715701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.716117 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.716119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.716265 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.716927 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.716940 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.718713 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.720500 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.720509 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.720511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.720855 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.721086 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.721087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.721088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.721246 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.721521 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.721529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.721723 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.721852 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.722055 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260807 22:40:04.723043 29000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.82.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.252@KRBTEST.COM" created.
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:40:04.730983 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.723319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38202)
0807 22:40:04.723365 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.723535 (+ 170us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.723580 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.723711 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.723712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.723858 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.723901 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.723903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.724785 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.724786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.725284 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.725285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.725863 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.725864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.725929 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.726350 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.726355 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.728016 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.729801 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.729802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.729803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.730039 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.730307 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.730307 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.730308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.730364 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.730746 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.730747 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.730786 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.730840 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.730913 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 22:40:04.731140 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.723399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.28.82.253:43145)
0807 22:40:04.723549 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.723554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.723578 (+ 24us) server_negotiation.cc:407] Connection header received
0807 22:40:04.723630 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.723632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.723645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.723712 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.723972 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.724691 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.725363 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.725775 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.725863 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.728106 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.728114 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.728114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.728121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.729712 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.730159 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.730162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.730163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.730230 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.730428 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.730429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.730430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.730662 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.730752 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.730893 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.730986 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260807 22:40:04.731891 29000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:04.732255 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.732126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59540)
0807 22:40:04.732174 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.732236 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 22:40:04.732409 28932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
I20260807 22:40:04.732837 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.732750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59556)
0807 22:40:04.732790 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.732815 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 22:40:04.732959 29000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41833: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab.
Entry for principal kudu/127.28.82.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab.
I20260807 22:40:04.744892 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.252:41833
--webserver_interface=127.28.82.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab
--principal=kudu/127.28.82.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:39737,127.28.82.253:43145,127.28.82.252:41833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:40:04.768674 28896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } attempt: 3
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:40:04.776036 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.769188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.28.82.253:43145)
0807 22:40:04.769276 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.769278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.769289 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:40:04.769355 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.769356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.769374 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.769428 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.769785 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.770280 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.771064 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.771293 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.771370 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.773392 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.773394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.773394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.773401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.774915 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.775314 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.775315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.775315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.775372 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.775568 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.775568 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.775569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.775675 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.775727 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.775894 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.775964 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 22:40:04.776082 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.769100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38204)
0807 22:40:04.769164 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.769178 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.769261 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.769450 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.769451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.769621 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.769691 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.769693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.770373 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.770375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.770976 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.770978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.771367 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.771368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.771450 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.771924 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.771929 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.773295 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.774997 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.774998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.774999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.775231 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.775435 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.775436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.775437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.775508 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.775735 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.775736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.775795 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.775869 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.775975 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 22:40:04.776573 28896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:04.776937 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.776831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59564)
0807 22:40:04.776879 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.776911 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:40:04.777245 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.777185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59576)
0807 22:40:04.777209 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.777232 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260807 22:40:04.777341 28896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41833: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
I20260807 22:40:04.782179 29000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } attempt: 1
I20260807 22:40:04.782629 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.782517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59586)
0807 22:40:04.782564 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.782601 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 22:40:04.782986 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.782896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59600)
0807 22:40:04.782934 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.782958 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 22:40:04.783066 29000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41833: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
I20260807 22:40:04.808496 28896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } attempt: 1
I20260807 22:40:04.809023 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.808879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59604)
0807 22:40:04.808947 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.808988 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 22:40:04.809381 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.809286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59614)
0807 22:40:04.809330 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.809361 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 22:40:04.809470 28896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41833: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
W20260807 22:40:04.833333 29015 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:40:04.833583 29015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:40:04.833612 29015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:40:04.833652 29015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:40:04.834896 29015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:40:04.834945 29015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:40:04.834960 29015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:40:04.834973 29015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:40:04.834986 29015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:40:04.836578 29015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:39737,127.28.82.253:43145,127.28.82.252:41833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab
--principal=kudu/127.28.82.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.252:41833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.82.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:40:04.836843 29015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:40:04.837090 29015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 22:40:04.842218 29015 server_base.cc:1034] running on GCE node
W20260807 22:40:04.842192 29029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:04.842186 29024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:04.842176 29025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:04.842700 29015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:40:04.843000 29015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:40:04.844149 29015 hybrid_clock.cc:648] HybridClock initialized: now 1786142404844121 us; error 42 us; skew 500 ppm
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:40:04.846787 29015 init.cc:377] Logged in from keytab as kudu/127.28.82.252@KRBTEST.COM (short username kudu)
I20260807 22:40:04.847278 29015 webserver.cc:460] Webserver started at http://127.28.82.252:39421/ using document root <none> and password file <none>
I20260807 22:40:04.847530 29015 fs_manager.cc:359] Metadata directory not provided
I20260807 22:40:04.847579 29015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:40:04.847748 29015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:40:04.848838 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/data/instance:
uuid: "08909944c28e495daa7129ab1ef14afe"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "fa10935a517c434df71a572dba9e587f"
server_key_iv: "76bb700548338ad5358ad476564905b6"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.849210 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/wal/instance:
uuid: "08909944c28e495daa7129ab1ef14afe"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "fa10935a517c434df71a572dba9e587f"
server_key_iv: "76bb700548338ad5358ad476564905b6"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.850610 29015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 22:40:04.851208 29000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } attempt: 2
I20260807 22:40:04.851428 29036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:40:04.851581 29015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 22:40:04.851616 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.851499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59630)
0807 22:40:04.851555 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.851586 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 22:40:04.851689 29015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/wal
uuid: "08909944c28e495daa7129ab1ef14afe"
format_stamp: "Formatted at 2026-08-07 22:40:04 on dist-test-slave-7kzw"
server_key: "fa10935a517c434df71a572dba9e587f"
server_key_iv: "76bb700548338ad5358ad476564905b6"
server_key_version: "encryptionkey@0"
I20260807 22:40:04.851785 29015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:40:04.852013 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.851921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59646)
0807 22:40:04.851969 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.851995 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:40:04.852105 29000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:41833: Network error: Client connection negotiation failed: client connection to 127.28.82.252:41833: connect: Connection refused (error 111)
I20260807 22:40:04.874102 29015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:40:04.876432 29015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:40:04.876612 29015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:40:04.881559 29015 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.252:41833
I20260807 22:40:04.881574 29113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.252:41833 every 8 connection(s)
I20260807 22:40:04.882125 29015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
I20260807 22:40:04.883324 29114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:40:04.884634 28896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } attempt: 2
I20260807 22:40:04.885310 29114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:40:04.891244 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29015
I20260807 22:40:04.891337 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786142120597695-29003-0/minicluster-data/master-2/wal/instance
I20260807 22:40:04.891561 29003 external_mini_cluster.cc:1428] Setting key d03ab9707b566967dd307d0790b47255
I20260807 22:40:04.895062 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.885018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59652)
0807 22:40:04.885095 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.885107 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.885158 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.886109 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.886111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.886397 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.886489 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.886493 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.887897 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.887899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.888462 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.888464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.889038 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.889040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.889088 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.889590 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.889596 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.891053 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.893436 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.893437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.893438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.893635 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.893928 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.893929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.893930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.894024 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.894609 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.894610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.894666 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.894791 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.894952 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":47}
I20260807 22:40:04.895364 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.886400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.28.82.254:39737)
0807 22:40:04.886443 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.886445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.886637 (+ 192us) server_negotiation.cc:407] Connection header received
0807 22:40:04.886680 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.886681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.886690 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.886738 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.887336 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.887840 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.888794 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.888979 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.889064 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.892006 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.892008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.892008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.892016 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.893920 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.894366 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.894367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.894368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.894434 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.894692 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.894693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.894693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.894897 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.894962 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.895173 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.895255 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":19}
I20260807 22:40:04.895408 29119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.885165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.28.82.252:41833)
0807 22:40:04.885376 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.885384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.885411 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:40:04.885888 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.885895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.885942 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.886121 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.886599 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.887820 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.888540 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.888969 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.889078 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.891142 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.891152 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.891152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.891164 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.893329 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.893744 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.893747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.893748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.893843 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.894108 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.894109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.894110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.894449 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.894896 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.894979 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.895117 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260807 22:40:04.895943 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.886353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:39737 (local address 127.0.0.1:56060)
0807 22:40:04.886518 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.886569 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.886632 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.886765 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.886767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.887112 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.887223 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.887252 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.887960 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.887961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.888714 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.888716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.889101 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.889102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.889169 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.889835 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.889844 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.891887 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.894000 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.894008 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.894010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.894283 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.894533 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.894534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.894535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.894621 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.894987 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.894995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.895046 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.895688 (+ 642us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.895828 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260807 22:40:04.896600 29114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:40:04.897955 28896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa: Bootstrap starting.
I20260807 22:40:04.898880 28896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa: Neither blocks nor log segments found. Creating new log.
I20260807 22:40:04.899271 28896 log.cc:824] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa: Log is configured to *not* fsync() on all Append() calls
I20260807 22:40:04.900301 28896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa: No bootstrap required, opened a new log
I20260807 22:40:04.901549 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.891940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.28.82.252:41833)
0807 22:40:04.892107 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.892109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.892226 (+ 117us) server_negotiation.cc:407] Connection header received
0807 22:40:04.892304 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.892305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.892316 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.892397 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.893221 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.893996 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.894952 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.895245 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.895305 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.897993 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.897994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.897995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.898003 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.900235 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.900620 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.900622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.900622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.900684 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.900894 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.900895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.900895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.901037 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.901196 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.901237 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.901328 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260807 22:40:04.903005 28896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } }
I20260807 22:40:04.903191 28896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a91c415ca9b4288a98af9cd44f345aa, State: Initialized, Role: FOLLOWER
I20260807 22:40:04.903331 28896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } }
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:40:04.903672 29127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.903744 28896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:40:04.903776 29127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:40:04.906038 29145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:40:04.906136 29145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:40:04.907596 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.898577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38214 (local address 127.28.82.253:43145)
0807 22:40:04.898656 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.898658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.898669 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:40:04.898699 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.898700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.898722 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.898784 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.899216 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.900201 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.901256 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.901581 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.901630 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.904296 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.904298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.904299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.904308 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.906300 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.906790 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.906791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.906791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.906846 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.907066 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.907067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.907068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.907181 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.907255 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.907425 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.907501 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:40:04.907596 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.896956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:43145 (local address 127.0.0.1:38214)
0807 22:40:04.897003 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.897014 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.897049 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.898847 (+ 1798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.898848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.899026 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.899104 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.899106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.900316 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.900317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.901138 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.901142 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.901857 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.901858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.901933 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.902364 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.902370 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.904165 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.906395 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.906396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.906396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.906684 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.906926 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.906927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.906927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.906995 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.907258 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.907259 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.907315 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.907397 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.907501 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:40:04.907042 28806 master_runner.cc:186] Error getting master registration for 127.28.82.253:43145: OK, instance_id { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" instance_seqno: 1786142404707312 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:40:04.910657 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.902273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56068 (local address 127.28.82.254:39737)
0807 22:40:04.902369 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.902371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.902384 (+ 13us) server_negotiation.cc:407] Connection header received
0807 22:40:04.902402 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.902402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.902410 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.902505 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.902982 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.903590 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.904386 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.904865 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.904949 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.908041 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.908043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.908043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.908050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.909433 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.909807 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.909808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.909809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.909870 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.910092 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.910093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.910093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.910209 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.910369 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.910446 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.910556 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 22:40:04.910828 28806 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:40:04.910816 29114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:40:04.919623 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.911076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59660)
0807 22:40:04.911124 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.911135 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.911171 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.911618 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.911619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.911787 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.911827 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.911829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.912817 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.912818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.913662 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.913664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.914150 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.914151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.914202 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.914608 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.914613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.916251 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.918406 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.918407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.918408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.918668 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.918872 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.918873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.918873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.918931 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.919278 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.919279 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.919321 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.919405 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.919512 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 22:40:04.919672 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.911439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.28.82.252:41833)
0807 22:40:04.911479 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.911481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.911488 (+ 7us) server_negotiation.cc:407] Connection header received
0807 22:40:04.911510 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.911510 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.911521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.911579 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.911910 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.912718 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.913756 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.914057 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.914149 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.916345 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.916347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.916347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.916354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.918309 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.918746 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.918747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.918748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.918811 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.918997 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.918998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.918998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.919211 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.919289 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.919428 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.919562 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":20}
I20260807 22:40:04.920089 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.911407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.28.82.253:43145)
0807 22:40:04.911441 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.911443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.911494 (+ 51us) server_negotiation.cc:407] Connection header received
0807 22:40:04.911563 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.911564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.911573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.911626 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.912081 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.912535 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.913183 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.913398 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.913471 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.916861 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.916862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.916862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.916870 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.918422 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.918916 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.918918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.918918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.918986 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.919188 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.919189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.919189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.919344 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.919521 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.919891 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.919980 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 22:40:04.921718 29114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe: Bootstrap starting.
I20260807 22:40:04.921928 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:40:04.922792 29114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe: Neither blocks nor log segments found. Creating new log.
I20260807 22:40:04.923178 29114 log.cc:824] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe: Log is configured to *not* fsync() on all Append() calls
I20260807 22:40:04.923071 28865 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8a91c415ca9b4288a98af9cd44f345aa"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56068
I20260807 22:40:04.923278 28865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 22:40:04.923338 28865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:40:04.924134 29114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe: No bootstrap required, opened a new log
I20260807 22:40:04.924320 28865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } }
I20260807 22:40:04.924574 28865 leader_election.cc:288] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [CANDIDATE]: Term 1 election: Requested vote from peers db521181b01245229ff3f84fa58aba4d (127.28.82.253:43145), 08909944c28e495daa7129ab1ef14afe (127.28.82.252:41833)
I20260807 22:40:04.926255 29114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } }
I20260807 22:40:04.926412 29114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08909944c28e495daa7129ab1ef14afe, State: Initialized, Role: FOLLOWER
I20260807 22:40:04.926563 29114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } }
I20260807 22:40:04.926895 29114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:40:04.926927 29076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a91c415ca9b4288a98af9cd44f345aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "08909944c28e495daa7129ab1ef14afe"
I20260807 22:40:04.927112 29076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:40:04.928501 29076 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a91c415ca9b4288a98af9cd44f345aa in term 1.
I20260807 22:40:04.928786 28830 leader_election.cc:302] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08909944c28e495daa7129ab1ef14afe, 8a91c415ca9b4288a98af9cd44f345aa; no voters:
I20260807 22:40:04.928987 29127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:40:04.929157 29127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [term 1 LEADER]: Becoming Leader. State: Replica: 8a91c415ca9b4288a98af9cd44f345aa, State: Running, Role: LEADER
I20260807 22:40:04.929270 29127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } }
I20260807 22:40:04.929771 29179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a91c415ca9b4288a98af9cd44f345aa. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.929925 29179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: This master's current role is: LEADER
W20260807 22:40:04.929289 29177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:40:04.930001 29177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:40:04.930686 29181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:40:04.930929 29181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:40:04.932968 29076 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [term 1 FOLLOWER]: Refusing update from remote peer 8a91c415ca9b4288a98af9cd44f345aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:40:04.933320 29127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:40:04.931910 29015 master_runner.cc:186] Error getting master registration for 127.28.82.253:43145: OK, instance_id { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" instance_seqno: 1786142404707312 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:40:04.935335 29015 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:40:04.935168 29159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a91c415ca9b4288a98af9cd44f345aa. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.935534 29159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:04.935755 29184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.935894 29184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:04.943238 29000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } attempt: 3
I20260807 22:40:04.944350 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.926680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.28.82.252:41833)
0807 22:40:04.927003 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.927005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.934998 (+ 7993us) server_negotiation.cc:407] Connection header received
0807 22:40:04.935808 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.935809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.935821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.935855 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.936756 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.937228 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.938277 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.938535 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.938605 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.940437 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.940438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.940439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.940446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.942403 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.943229 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.943231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.943231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.943298 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.943570 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.943571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.943572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.943724 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.943833 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.944142 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.944230 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299}
I20260807 22:40:04.946427 29182 mvcc.cc:205] Tried to move back new op lower bound from 7316039290602119168 to 7316039290591219712. Current Snapshot: MvccSnapshot[applied={T|T < 7316039290602119168}]
I20260807 22:40:04.948016 29127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.948132 29127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: This master's current role is: LEADER
I20260807 22:40:04.948323 29184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.948526 29184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:04.949626 29186 mvcc.cc:205] Tried to move back new op lower bound from 7316039290602119168 to 7316039290591219712. Current Snapshot: MvccSnapshot[applied={T|T < 7316039290602119168 or (T in {7316039290602119168})}]
Aug 07 22:40:04 dist-test-slave-7kzw krb5kdc[28785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:40:04.952098 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.925736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.28.82.253:43145)
0807 22:40:04.925787 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.925789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.934827 (+ 9038us) server_negotiation.cc:407] Connection header received
0807 22:40:04.934948 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.934949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.934962 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.934997 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.938565 (+ 3568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.939372 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.940498 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.940847 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.940896 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.947034 (+ 6138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.947035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.947036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.947043 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.948877 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.951176 (+ 2299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.951178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.951178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.951246 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.951489 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.951490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.951491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.951648 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.951775 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.951899 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.951984 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:40:04.954854 29119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.943847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.28.82.252:41833)
0807 22:40:04.943889 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.943891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.943899 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:40:04.943919 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.943920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.943929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.943985 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.944327 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.945138 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.946254 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.946632 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.946696 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.950827 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.950828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.950829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.950837 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.952549 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.953882 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.953884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.953884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.953945 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.954149 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.954150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.954151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.954283 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.954362 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.954641 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.954745 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:40:04.955371 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.943593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:41833 (local address 127.0.0.1:59676)
0807 22:40:04.943702 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:04.943722 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:04.943765 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:04.944004 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:04.944006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:04.944189 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:04.944232 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.944234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.945243 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.945245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.946144 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:04.946146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.946762 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.946764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:04.946829 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.949063 (+ 2234us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:04.949069 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:04.950729 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:04.952623 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.952624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.952625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.952996 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.954018 (+ 1022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:04.954019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:04.954019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:04.954091 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:04.954366 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:04.954367 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:04.954412 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 22:40:04.954542 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 22:40:04.955272 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 22:40:04.956007 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:04.930377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.28.82.254:39737)
0807 22:40:04.930423 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:04.930425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:04.938664 (+ 8239us) server_negotiation.cc:407] Connection header received
0807 22:40:04.939301 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:04.939304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:04.939318 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:04.939351 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:04.941152 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.941886 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.942914 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:04.943215 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:04.943298 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:04.950063 (+ 6765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:04.950064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:04.950065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:04.950073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:04.953807 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.954693 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.954694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.954694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.954783 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:04.955293 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:04.955294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:04.955295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:04.955458 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:04.955577 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:04.955827 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 22:40:04.955906 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 22:40:04.956509 29181 catalog_manager.cc:1347] Generated new cluster ID: 4fbfc1743f41470190dd838e692c1df7
I20260807 22:40:04.956558 29181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:40:04.957582 29000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d: Bootstrap starting.
I20260807 22:40:04.958279 29000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d: Neither blocks nor log segments found. Creating new log.
I20260807 22:40:04.958587 29000 log.cc:824] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d: Log is configured to *not* fsync() on all Append() calls
I20260807 22:40:04.967001 29181 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:40:04.968158 29181 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:40:04.968448 29000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d: No bootstrap required, opened a new log
I20260807 22:40:04.970424 29000 raft_consensus.cc:348] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } }
I20260807 22:40:04.970615 29000 raft_consensus.cc:729] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db521181b01245229ff3f84fa58aba4d, State: Initialized, Role: FOLLOWER
I20260807 22:40:04.970692 29000 consensus_queue.cc:260] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } }
I20260807 22:40:04.971468 28966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:40:04.972537 28966 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [term 1 FOLLOWER]: Refusing update from remote peer 8a91c415ca9b4288a98af9cd44f345aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:40:04.972847 29127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:40:04.971491 28967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a91c415ca9b4288a98af9cd44f345aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "db521181b01245229ff3f84fa58aba4d"
I20260807 22:40:04.974059 28967 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a91c415ca9b4288a98af9cd44f345aa in term 1.
I20260807 22:40:04.971374 29000 sys_catalog.cc:564] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:40:04.975714 29181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa: Generated new TSK 0
I20260807 22:40:04.975932 29181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 22:40:04.976234 29199 sys_catalog.cc:455] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.976706 29199 sys_catalog.cc:458] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:04.976785 29199 sys_catalog.cc:455] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a91c415ca9b4288a98af9cd44f345aa. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.976886 29199 sys_catalog.cc:458] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:04.976924 29199 sys_catalog.cc:455] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.976959 29199 sys_catalog.cc:458] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:40:04.977195 29221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:40:04.977286 29221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:40:04.978001 29201 mvcc.cc:205] Tried to move back new op lower bound from 7316039290738253824 to 7316039290591219712. Current Snapshot: MvccSnapshot[applied={T|T < 7316039290602119168}]
I20260807 22:40:04.978950 29194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a91c415ca9b4288a98af9cd44f345aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a91c415ca9b4288a98af9cd44f345aa" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 39737 } } peers { permanent_uuid: "db521181b01245229ff3f84fa58aba4d" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 43145 } } peers { permanent_uuid: "08909944c28e495daa7129ab1ef14afe" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 41833 } } }
I20260807 22:40:04.979045 29194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a91c415ca9b4288a98af9cd44f345aa [sys.catalog]: This master's current role is: LEADER
W20260807 22:40:05.008194 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.006318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.28.82.253:43145)
0807 22:40:05.006369 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.006372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.007445 (+ 1073us) server_negotiation.cc:407] Connection header received
0807 22:40:05.007566 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.007567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.007580 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.007615 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.008114 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38238: BlockingRecv error: recv got EOF from 127.0.0.1:38238 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 22:40:05.009060 29119 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.006795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.28.82.252:41833)
0807 22:40:05.006852 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.006854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.008486 (+ 1632us) server_negotiation.cc:407] Connection header received
0807 22:40:05.008594 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.008596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.008607 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.008640 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.008989 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59684: BlockingRecv error: recv got EOF from 127.0.0.1:59684 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260807 22:40:05.010000 28899 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.007122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56094 (local address 127.28.82.254:39737)
0807 22:40:05.007166 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.007168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.009317 (+ 2149us) server_negotiation.cc:407] Connection header received
0807 22:40:05.009486 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.009488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.009499 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.009532 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.009931 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56094: BlockingRecv error: recv got EOF from 127.0.0.1:56094 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 22:40:05.012712 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.011891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.28.82.253:43145)
0807 22:40:05.011929 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.011931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.012134 (+ 203us) server_negotiation.cc:407] Connection header received
0807 22:40:05.012268 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.012269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.012293 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.012331 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.012659 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38240: BlockingRecv error: recv got EOF from 127.0.0.1:38240 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 22:40:05.015961 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.015106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.28.82.253:43145)
0807 22:40:05.015143 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.015145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.015431 (+ 286us) server_negotiation.cc:407] Connection header received
0807 22:40:05.015534 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.015535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.015546 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.015577 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.015912 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38256: BlockingRecv error: recv got EOF from 127.0.0.1:38256 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 22:40:05.019927 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.019093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38262 (local address 127.28.82.253:43145)
0807 22:40:05.019143 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.019145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.019410 (+ 265us) server_negotiation.cc:407] Connection header received
0807 22:40:05.019512 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.019513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.019523 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.019554 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.019881 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38262: BlockingRecv error: recv got EOF from 127.0.0.1:38262 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 22:40:05.025900 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.025075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.28.82.253:43145)
0807 22:40:05.025112 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.025114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.025375 (+ 261us) server_negotiation.cc:407] Connection header received
0807 22:40:05.025476 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.025477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.025487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.025517 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.025850 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38276: BlockingRecv error: recv got EOF from 127.0.0.1:38276 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 22:40:05.036285 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.035232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.28.82.253:43145)
0807 22:40:05.035277 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.035280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.035602 (+ 322us) server_negotiation.cc:407] Connection header received
0807 22:40:05.035723 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.035725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.035737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.035773 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.036225 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38288: BlockingRecv error: recv got EOF from 127.0.0.1:38288 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 22:40:05.055091 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.053795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.28.82.253:43145)
0807 22:40:05.053843 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.053845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.054232 (+ 387us) server_negotiation.cc:407] Connection header received
0807 22:40:05.054412 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.054414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.054427 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.054526 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.055023 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38292: BlockingRecv error: recv got EOF from 127.0.0.1:38292 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 22:40:05.089812 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.088664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.28.82.253:43145)
0807 22:40:05.088718 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.088720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.089100 (+ 380us) server_negotiation.cc:407] Connection header received
0807 22:40:05.089236 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.089238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.089252 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.089287 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.089744 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38304: BlockingRecv error: recv got EOF from 127.0.0.1:38304 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 22:40:05.156213 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.155243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.28.82.253:43145)
0807 22:40:05.155315 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.155317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.155430 (+ 113us) server_negotiation.cc:407] Connection header received
0807 22:40:05.155519 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.155520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.155531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.155667 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.156133 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38320: BlockingRecv error: recv got EOF from 127.0.0.1:38320 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 22:40:05.286401 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.285569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.28.82.253:43145)
0807 22:40:05.285633 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.285635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.285661 (+ 26us) server_negotiation.cc:407] Connection header received
0807 22:40:05.285781 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.285782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.285810 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.285876 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.286301 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38328: BlockingRecv error: recv got EOF from 127.0.0.1:38328 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260807 22:40:05.544674 29012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:05.543830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.28.82.253:43145)
0807 22:40:05.543914 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:05.543917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:05.544030 (+ 113us) server_negotiation.cc:407] Connection header received
0807 22:40:05.544142 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:05.544144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:05.544155 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:05.544226 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:05.544621 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38344: BlockingRecv error: recv got EOF from 127.0.0.1:38344 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:40:05.930634 29177 catalog_manager.cc:1259] Loaded cluster ID: 4fbfc1743f41470190dd838e692c1df7
I20260807 22:40:05.930737 29177 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe: loading cluster ID for follower catalog manager: success
I20260807 22:40:05.931553 29177 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe: acquiring CA information for follower catalog manager: success
I20260807 22:40:05.931900 29177 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 08909944c28e495daa7129ab1ef14afe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:40:05.977771 29221 catalog_manager.cc:1259] Loaded cluster ID: 4fbfc1743f41470190dd838e692c1df7
I20260807 22:40:05.977886 29221 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d: loading cluster ID for follower catalog manager: success
I20260807 22:40:05.978814 29221 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d: acquiring CA information for follower catalog manager: success
I20260807 22:40:05.979137 29221 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db521181b01245229ff3f84fa58aba4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:40:06.060806 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:06.058165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.28.82.253:43145)
0807 22:40:06.058335 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:06.058337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:06.058364 (+ 27us) server_negotiation.cc:407] Connection header received
0807 22:40:06.058444 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:06.058445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:06.058523 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:06.058605 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:06.058738 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.059296 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.060032 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.060256 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.060313 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:06.060443 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:06.060563 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:06.060625 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:06.060661 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 22:40:06.060679 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260807 22:40:06.065075 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:06.062826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59696 (local address 127.28.82.252:41833)
0807 22:40:06.062926 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:06.062928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:06.062936 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:40:06.063005 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:06.063006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:06.063042 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:06.063106 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:06.063186 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.063647 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.064387 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.064571 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.064621 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:06.064759 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:06.064846 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:06.064896 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:06.064943 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 22:40:06.064958 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":50,"threads_started":1}
I20260807 22:40:06.069815 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:06.066943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.28.82.253:43145)
0807 22:40:06.067051 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:06.067054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:06.067289 (+ 235us) server_negotiation.cc:407] Connection header received
0807 22:40:06.067401 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:06.067401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:06.067412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:06.067438 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:06.067654 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.068366 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.069104 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.069303 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.069376 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:06.069548 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:06.069585 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:06.069643 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:06.069705 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 22:40:06.069729 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 22:40:06.070382 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:06.067149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.28.82.252:41833)
0807 22:40:06.067184 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:06.067186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:06.067194 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:40:06.067290 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:06.067291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:06.067301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:06.067361 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:06.067491 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.068214 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.069510 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.069822 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.069889 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:06.070099 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:06.070130 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:06.070163 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:06.070271 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 22:40:06.070294 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 22:40:06.070572 29342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:06.066957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56100 (local address 127.28.82.254:39737)
0807 22:40:06.067085 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:06.067087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:06.067175 (+ 88us) server_negotiation.cc:407] Connection header received
0807 22:40:06.067320 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:06.067321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:06.067376 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:06.067440 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:06.067650 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.068185 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.069467 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:06.069789 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:06.069858 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:06.070105 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:06.070228 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:06.070279 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:06.070421 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 22:40:06.070457 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260807 22:40:06.072377 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 28806
I20260807 22:40:06.087009 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 28906
I20260807 22:40:06.097527 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29015
2026-08-07T22:40:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3038 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 22:40:06 dist-test-slave-7kzw krb5kdc[29347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 22:40:06 dist-test-slave-7kzw krb5kdc[29347](info): set up 2 sockets
Aug 07 22:40:06 dist-test-slave-7kzw krb5kdc[29347](info): commencing operation
krb5kdc: starting...
W20260807 22:40:07.399611 29003 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/test-user.keytab.
Aug 07 22:40:07 dist-test-slave-7kzw krb5kdc[29347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T22:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T22:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.82.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
Entry for principal kudu/127.28.82.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab.
I20260807 22:40:07.531484 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45079
--webserver_interface=127.28.82.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:33735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:45079,127.28.82.253:38573,127.28.82.252:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 22:40:07.616858 29364 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:40:07.617167 29364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:40:07.617200 29364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:40:07.617249 29364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:40:07.618512 29364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:40:07.618595 29364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:40:07.618618 29364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:40:07.618639 29364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:40:07.618670 29364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:40:07.620312 29364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:45079,127.28.82.253:38573,127.28.82.252:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.254.keytab
--principal=kudu/127.28.82.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.254:45079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.82.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:40:07.620636 29364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:40:07.620893 29364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:40:07.626082 29372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:07.626144 29373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:07.626096 29377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:07.626214 29364 server_base.cc:1034] running on GCE node
I20260807 22:40:07.626673 29364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:40:07.626956 29364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:40:07.628113 29364 hybrid_clock.cc:648] HybridClock initialized: now 1786142407628092 us; error 34 us; skew 500 ppm
Aug 07 22:40:07 dist-test-slave-7kzw krb5kdc[29347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:40:07.635778 29364 init.cc:377] Logged in from keytab as kudu/127.28.82.254@KRBTEST.COM (short username kudu)
I20260807 22:40:07.636289 29364 webserver.cc:460] Webserver started at http://127.28.82.254:40793/ using document root <none> and password file <none>
I20260807 22:40:07.636549 29364 fs_manager.cc:359] Metadata directory not provided
I20260807 22:40:07.636600 29364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:40:07.636729 29364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:40:07.638151 29364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/data/instance:
uuid: "78fdaae4e2bf422091cb5128a2f276e9"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "ebe22e52f23d45ba9e5c5242aac01ceb"
server_key_iv: "0a3a4c55c808f966a645989c4f9b2cda"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.638592 29364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/wal/instance:
uuid: "78fdaae4e2bf422091cb5128a2f276e9"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "ebe22e52f23d45ba9e5c5242aac01ceb"
server_key_iv: "0a3a4c55c808f966a645989c4f9b2cda"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.640014 29364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:40:07.640825 29386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:40:07.641012 29364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:40:07.641129 29364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/wal
uuid: "78fdaae4e2bf422091cb5128a2f276e9"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "ebe22e52f23d45ba9e5c5242aac01ceb"
server_key_iv: "0a3a4c55c808f966a645989c4f9b2cda"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.641204 29364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:40:07.655155 29364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:40:07.657166 29364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:40:07.657342 29364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:40:07.662210 29364 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.254:45079
I20260807 22:40:07.662284 29457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.254:45079 every 8 connection(s)
I20260807 22:40:07.662748 29364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/data/info.pb
I20260807 22:40:07.664232 29458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:40:07.666509 29458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:07.667774 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29364
I20260807 22:40:07.667898 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-0/wal/instance
I20260807 22:40:07.668107 29003 external_mini_cluster.cc:1428] Setting key c1c80478d8176f90b476786880ea36c1
Aug 07 22:40:07 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.82.253@KRBTEST.COM; defaulting to no policy
I20260807 22:40:07.678915 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.667390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48262 (local address 127.28.82.254:45079)
0807 22:40:07.667930 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:07.667935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:07.667977 (+ 42us) server_negotiation.cc:407] Connection header received
0807 22:40:07.668535 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:07.668538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:07.668560 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:07.668660 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:07.669486 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.670854 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.671974 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.672493 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.672618 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:07.675368 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:07.675376 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:07.675377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:07.675383 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:07.677037 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:07.677533 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:07.677536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:07.677536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:07.677651 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:07.677924 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:07.677925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:07.677925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:07.678206 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:07.678273 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:07.678610 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 22:40:07.678756 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":478,"thread_start_us":120,"threads_started":1}
I20260807 22:40:07.678925 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.667388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45079 (local address 127.0.0.1:48262)
0807 22:40:07.667586 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.667635 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:07.668313 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:07.668716 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:07.668719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:07.669151 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:07.669382 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.669388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.670950 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.670952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:07.671882 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.671885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.672660 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.672661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:07.672720 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:07.673358 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:07.673373 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:07.675275 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:07.677140 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:07.677148 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:07.677151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:07.677445 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:07.677729 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:07.677730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:07.677730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:07.677856 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:07.678279 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:07.678288 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:07.678373 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 22:40:07.678537 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 22:40:07.678687 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260807 22:40:07.679725 29458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:07.680119 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.679970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56388)
0807 22:40:07.680043 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.680089 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 22:40:07.680274 29390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.253@KRBTEST.COM" created.
I20260807 22:40:07.680850 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.680725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56404)
0807 22:40:07.680800 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.680830 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 22:40:07.680979 29458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:38573: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab.
Entry for principal kudu/127.28.82.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab.
I20260807 22:40:07.697196 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.253:38573
--webserver_interface=127.28.82.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab
--principal=kudu/127.28.82.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:33735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:45079,127.28.82.253:38573,127.28.82.252:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:40:07.730238 29458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } attempt: 1
I20260807 22:40:07.730894 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.730722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56420)
0807 22:40:07.730806 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.730858 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 22:40:07.731369 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.731229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56432)
0807 22:40:07.731302 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.731339 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 22:40:07.731501 29458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:38573: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
W20260807 22:40:07.786296 29468 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:40:07.786851 29468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:40:07.786896 29468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:40:07.786922 29468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:40:07.788236 29468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:40:07.788290 29468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:40:07.788304 29468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:40:07.788317 29468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:40:07.788329 29468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:40:07.789899 29468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:45079,127.28.82.253:38573,127.28.82.252:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.253.keytab
--principal=kudu/127.28.82.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.253:38573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.82.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:40:07.790127 29468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:40:07.790333 29468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:40:07.795807 29477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:07.795866 29475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:07.795994 29468 server_base.cc:1034] running on GCE node
W20260807 22:40:07.795866 29479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:07.796408 29468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:40:07.796809 29468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:40:07.798026 29468 hybrid_clock.cc:648] HybridClock initialized: now 1786142407797994 us; error 52 us; skew 500 ppm
I20260807 22:40:07.799716 29458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } attempt: 2
I20260807 22:40:07.800390 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.800232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56442)
0807 22:40:07.800314 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.800354 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 22:40:07.800784 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.800673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56452)
0807 22:40:07.800720 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.800750 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 22:40:07.800877 29458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.253:38573: Network error: Client connection negotiation failed: client connection to 127.28.82.253:38573: connect: Connection refused (error 111)
Aug 07 22:40:07 dist-test-slave-7kzw krb5kdc[29347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:40:07.801414 29468 init.cc:377] Logged in from keytab as kudu/127.28.82.253@KRBTEST.COM (short username kudu)
I20260807 22:40:07.801970 29468 webserver.cc:460] Webserver started at http://127.28.82.253:36193/ using document root <none> and password file <none>
I20260807 22:40:07.802225 29468 fs_manager.cc:359] Metadata directory not provided
I20260807 22:40:07.802295 29468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:40:07.802459 29468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:40:07.803627 29468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/data/instance:
uuid: "df04188ab64e40da8fabeb0c300d1fc2"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "95dddfe61017ec89aa28bf63fe449231"
server_key_iv: "bde33e57ddb11ef3fb1dc6ea66ef742c"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.804050 29468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/wal/instance:
uuid: "df04188ab64e40da8fabeb0c300d1fc2"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "95dddfe61017ec89aa28bf63fe449231"
server_key_iv: "bde33e57ddb11ef3fb1dc6ea66ef742c"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.805809 29468 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 22:40:07.806758 29488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:40:07.806954 29468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 22:40:07.807070 29468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/wal
uuid: "df04188ab64e40da8fabeb0c300d1fc2"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "95dddfe61017ec89aa28bf63fe449231"
server_key_iv: "bde33e57ddb11ef3fb1dc6ea66ef742c"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.807152 29468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:40:07.817314 29468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:40:07.819540 29468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:40:07.819710 29468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:40:07.824266 29468 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.253:38573
I20260807 22:40:07.824312 29555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.253:38573 every 8 connection(s)
I20260807 22:40:07.824707 29468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/data/info.pb
I20260807 22:40:07.825716 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:40:07.827484 29556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 22:40:07 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:40:07.833673 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29468
I20260807 22:40:07.833799 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-1/wal/instance
I20260807 22:40:07.834017 29003 external_mini_cluster.cc:1428] Setting key bff7f5cc3a3dc6a380029549d46eb81b
I20260807 22:40:07.837088 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.828307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48274 (local address 127.28.82.254:45079)
0807 22:40:07.828375 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:07.828377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:07.828607 (+ 230us) server_negotiation.cc:407] Connection header received
0807 22:40:07.829060 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:07.829062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:07.829075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:07.829135 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:07.829750 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.830259 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.831298 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.831515 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.831627 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:07.834168 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:07.834170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:07.834170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:07.834178 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:07.835591 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:07.836032 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:07.836033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:07.836034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:07.836096 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:07.836363 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:07.836364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:07.836365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:07.836487 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:07.836588 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:07.836945 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 22:40:07.837019 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 22:40:07.837423 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.828288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45079 (local address 127.0.0.1:48274)
0807 22:40:07.828462 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.828499 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:07.828974 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:07.829201 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:07.829204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:07.829467 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:07.829670 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.829674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.830342 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.830343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:07.831206 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.831209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.831619 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.831620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:07.831719 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:07.832299 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:07.832308 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:07.834058 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:07.835700 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:07.835709 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:07.835711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:07.835951 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:07.836179 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:07.836180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:07.836181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:07.836298 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:07.836602 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:07.836611 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:07.836796 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 22:40:07.836930 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 22:40:07.837127 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260807 22:40:07.838192 29556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.82.252@KRBTEST.COM; defaulting to no policy
Aug 07 22:40:07 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.82.252@KRBTEST.COM" created.
I20260807 22:40:07.846513 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.838590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56464 (local address 127.28.82.253:38573)
0807 22:40:07.838804 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:07.838808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:07.838827 (+ 19us) server_negotiation.cc:407] Connection header received
0807 22:40:07.838846 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:07.838847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:07.838860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:07.838950 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:07.839265 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.840030 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.840778 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.841118 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.841202 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:07.843230 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:07.843239 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:07.843239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:07.843256 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:07.844957 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:07.845371 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:07.845374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:07.845375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:07.845464 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:07.845691 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:07.845693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:07.845693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:07.845920 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:07.846022 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:07.846219 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 22:40:07.846365 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":185,"thread_start_us":110,"threads_started":1}
I20260807 22:40:07.846522 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.838547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56464)
0807 22:40:07.838597 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.838607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:07.838652 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:07.838950 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:07.838951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:07.839118 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:07.839176 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.839177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.840137 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.840138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:07.840689 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.840691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.841211 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.841212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:07.841295 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:07.841759 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:07.841765 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:07.843137 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:07.845058 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:07.845059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:07.845059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:07.845284 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:07.845539 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:07.845540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:07.845541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:07.845610 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:07.846000 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:07.846001 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:07.846073 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 22:40:07.846165 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 22:40:07.846310 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 22:40:07.847299 29556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:07.847723 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.847577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53730)
0807 22:40:07.847627 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.847684 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 22:40:07.847868 29489 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
I20260807 22:40:07.848333 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.848221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53746)
0807 22:40:07.848267 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.848302 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 22:40:07.848508 29556 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:39625: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.82.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab.
Entry for principal kudu/127.28.82.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab.
I20260807 22:40:07.861541 29003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskj8JEAI/build/release/bin/kudu
/tmp/dist-test-taskj8JEAI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.252:39625
--webserver_interface=127.28.82.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab
--principal=kudu/127.28.82.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.82.212:33735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.82.254:45079,127.28.82.253:38573,127.28.82.252:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 22:40:07.892031 29458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } attempt: 3
Aug 07 22:40:07 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:40:07.897814 29556 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } attempt: 1
I20260807 22:40:07.898281 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.898123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53750)
0807 22:40:07.898190 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.898241 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 22:40:07.898644 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.898565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53764)
0807 22:40:07.898599 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.898621 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 22:40:07.898730 29556 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:39625: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
I20260807 22:40:07.899752 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.892535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.28.82.253:38573)
0807 22:40:07.892612 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:07.892614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:07.892625 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:40:07.892655 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:07.892656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:07.892668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:07.892739 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:07.893112 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.893642 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.894345 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.894633 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.894712 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:07.896827 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:07.896833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:07.896833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:07.896841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:07.898504 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:07.898955 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:07.898956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:07.898957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:07.899020 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:07.899219 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:07.899220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:07.899220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:07.899345 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:07.899419 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:07.899574 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 22:40:07.899658 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 22:40:07.899753 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.892452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56468)
0807 22:40:07.892519 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.892529 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:07.892592 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:07.892776 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:07.892778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:07.892953 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:07.893024 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.893026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.893730 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.893731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:07.894260 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.894262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:07.894711 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:07.894712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:07.894815 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:07.895341 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:07.895346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:07.896728 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:07.898607 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:07.898608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:07.898609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:07.898865 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:07.899086 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:07.899087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:07.899087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:07.899156 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:07.899414 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:07.899415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:07.899466 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 22:40:07.899548 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 22:40:07.899656 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 22:40:07.900154 29458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:07.900511 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.900410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53778)
0807 22:40:07.900457 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.900484 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 22:40:07.900846 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.900749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53794)
0807 22:40:07.900794 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.900817 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 22:40:07.901010 29458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:39625: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
I20260807 22:40:07.932233 29458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } attempt: 1
I20260807 22:40:07.932852 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.932646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53798)
0807 22:40:07.932734 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.932798 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 22:40:07.933357 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.933216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53808)
0807 22:40:07.933296 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.933331 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 22:40:07.933498 29458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:39625: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
W20260807 22:40:07.952894 29567 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 22:40:07.953181 29567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 22:40:07.953231 29567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 22:40:07.953290 29567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 22:40:07.954545 29567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 22:40:07.954610 29567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 22:40:07.954632 29567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 22:40:07.954654 29567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 22:40:07.954682 29567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 22:40:07.956280 29567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.82.212:33735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.82.254:45079,127.28.82.253:38573,127.28.82.252:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/krb5kdc/kudu_127.28.82.252.keytab
--principal=kudu/127.28.82.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.82.252:39625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.82.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 21:52:04 UTC on ccycloud.quasar-kxqltd.root.comops.site
build id 4573644
I20260807 22:40:07.956583 29567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 22:40:07.956842 29567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 22:40:07.962113 29576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 22:40:07.962145 29580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:07.962375 29567 server_base.cc:1034] running on GCE node
W20260807 22:40:07.962505 29574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 22:40:07.962818 29567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 22:40:07.963174 29567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 22:40:07.964342 29567 hybrid_clock.cc:648] HybridClock initialized: now 1786142407964336 us; error 50 us; skew 500 ppm
Aug 07 22:40:07 dist-test-slave-7kzw krb5kdc[29347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 22:40:07.966858 29556 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } attempt: 2
I20260807 22:40:07.967082 29567 init.cc:377] Logged in from keytab as kudu/127.28.82.252@KRBTEST.COM (short username kudu)
I20260807 22:40:07.967281 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.967132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53820)
0807 22:40:07.967194 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.967228 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 22:40:07.967684 29567 webserver.cc:460] Webserver started at http://127.28.82.252:40073/ using document root <none> and password file <none>
I20260807 22:40:07.967751 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.967559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53828)
0807 22:40:07.967633 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.967700 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 22:40:07.967905 29556 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.82.252:39625: Network error: Client connection negotiation failed: client connection to 127.28.82.252:39625: connect: Connection refused (error 111)
I20260807 22:40:07.967971 29567 fs_manager.cc:359] Metadata directory not provided
I20260807 22:40:07.968037 29567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 22:40:07.968174 29567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 22:40:07.969311 29567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/data/instance:
uuid: "9d83545c451d4401b7aa6382da5199e5"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "401e9b11c8292654218e01cbfb965f2c"
server_key_iv: "25a84dd7995270465c16d0c1df07cbdf"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.969704 29567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/wal/instance:
uuid: "9d83545c451d4401b7aa6382da5199e5"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "401e9b11c8292654218e01cbfb965f2c"
server_key_iv: "25a84dd7995270465c16d0c1df07cbdf"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.971268 29567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 22:40:07.972113 29590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 22:40:07.972345 29567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 22:40:07.972468 29567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/data,/tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/wal
uuid: "9d83545c451d4401b7aa6382da5199e5"
format_stamp: "Formatted at 2026-08-07 22:40:07 on dist-test-slave-7kzw"
server_key: "401e9b11c8292654218e01cbfb965f2c"
server_key_iv: "25a84dd7995270465c16d0c1df07cbdf"
server_key_version: "encryptionkey@0"
I20260807 22:40:07.972548 29567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 22:40:07.987932 29567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 22:40:07.990093 29567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 22:40:07.990281 29567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 22:40:07.994445 29567 rpc_server.cc:307] RPC server started. Bound to: 127.28.82.252:39625
I20260807 22:40:07.994518 29658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.82.252:39625 every 8 connection(s)
I20260807 22:40:07.994938 29567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/data/info.pb
I20260807 22:40:07.995944 29659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 22:40:07.997591 29659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:07.997741 29003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskj8JEAI/build/release/bin/kudu as pid 29567
I20260807 22:40:07.997900 29003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskj8JEAI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786142120597695-29003-0/minicluster-data/master-2/wal/instance
I20260807 22:40:07.998140 29003 external_mini_cluster.cc:1428] Setting key 6a34b13be2030c7e0ba42be1d1bc7506
Aug 07 22:40:08 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
Aug 07 22:40:08 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:40:08.007681 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.998428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.28.82.254:45079)
0807 22:40:07.998521 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:07.998523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:07.998747 (+ 224us) server_negotiation.cc:407] Connection header received
0807 22:40:07.999318 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:07.999319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:07.999331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:07.999397 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:07.999982 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.000468 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.001352 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.001548 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.001626 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.004184 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.004185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.004185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.004194 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.006044 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.006550 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.006552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.006552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.006623 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.006899 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.006900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.006900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.007036 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.007088 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.007471 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.007569 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 22:40:08.007778 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.998410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.254:45079 (local address 127.0.0.1:48290)
0807 22:40:07.998612 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:07.998656 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:07.999187 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:07.999424 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:07.999427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:07.999713 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:07.999883 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:07.999887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.000552 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.000553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.001257 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.001259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.001654 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.001655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.001801 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.002382 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:08.002394 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:08.004089 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:08.006138 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.006145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.006147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.006416 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.006698 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.006698 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.006699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.006819 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.007102 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:08.007111 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:08.007298 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 22:40:08.007411 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 22:40:08.007569 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260807 22:40:08.008411 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:07.998658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53842 (local address 127.28.82.252:39625)
0807 22:40:07.998831 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:07.998837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:07.998873 (+ 36us) server_negotiation.cc:407] Connection header received
0807 22:40:07.999202 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:07.999213 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:07.999235 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:07.999340 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:07.999999 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.000813 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.001554 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.001911 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.001988 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.004701 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.004709 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.004709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.004715 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.006819 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.007189 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.007191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.007192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.007245 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.007462 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.007463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.007464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.007644 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.007806 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.008197 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.008288 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260807 22:40:08.008392 29659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:08.008642 29458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } attempt: 2
Aug 07 22:40:08 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.254@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
Aug 07 22:40:08 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
Aug 07 22:40:08 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.254@KRBTEST.COM
I20260807 22:40:08.016889 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.008922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53844)
0807 22:40:08.008998 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:08.009007 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:08.009045 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:08.009170 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:08.009171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:08.009341 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:08.009390 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.009392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.010229 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.010230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.010897 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.010899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.011371 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.011372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.011428 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.011885 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:08.011891 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:08.013346 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:08.015194 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.015195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.015196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.015367 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.015570 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.015571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.015571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.015633 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.015939 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:08.015940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:08.015999 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 22:40:08.016587 (+ 588us) client_negotiation.cc:240] Negotiation successful
0807 22:40:08.016713 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 22:40:08.017045 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.008983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.28.82.252:39625)
0807 22:40:08.009040 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.009042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.009050 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:40:08.009094 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.009094 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.009106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.009154 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.009471 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.010168 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.010974 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.011277 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.011593 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.013450 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.013451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.013452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.013460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.015095 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.015440 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.015441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.015442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.015497 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.015707 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.015708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.015708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.015854 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.015931 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.016840 (+ 909us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.016929 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 22:40:08.018306 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.008664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.253:38573 (local address 127.0.0.1:56478)
0807 22:40:08.008699 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:08.008710 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:08.008752 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:08.008948 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:08.008950 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:08.009116 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:08.009178 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.009180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.010163 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.010164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.011025 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.011028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.011521 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.011523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.011600 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.012375 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:08.012380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:08.013945 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:08.017296 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.017298 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.017298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.017513 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.017724 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.017724 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.017725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.017779 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.018070 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:08.018070 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:08.018106 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 22:40:08.018157 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 22:40:08.018232 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 22:40:08.018376 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.008792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.28.82.253:38573)
0807 22:40:08.008838 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.008840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.008848 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:40:08.008875 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.008875 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.008887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.008946 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.009307 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.010083 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.011134 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.011440 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.011510 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.014044 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.014046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.014047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.014056 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.017204 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.017579 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.017581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.017581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.017650 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.017845 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.017846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.017847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.017994 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.018077 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.018186 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.018271 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:40:08.018813 29659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } has no permanent_uuid. Determining permanent_uuid...
I20260807 22:40:08.019058 29458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9: Bootstrap starting.
I20260807 22:40:08.019943 29458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9: Neither blocks nor log segments found. Creating new log.
I20260807 22:40:08.020337 29458 log.cc:824] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9: Log is configured to *not* fsync() on all Append() calls
I20260807 22:40:08.021158 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.009663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48298 (local address 127.28.82.254:45079)
0807 22:40:08.009727 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.009728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.009772 (+ 44us) server_negotiation.cc:407] Connection header received
0807 22:40:08.009833 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.009833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.009842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.009890 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.010567 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.011230 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.012020 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.012221 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.012296 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.014830 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.014831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.014831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.014839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.016431 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.016769 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.016770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.016771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.016848 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.017437 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.017438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.017438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.020726 (+ 3288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.020890 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.020986 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.021073 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 22:40:08.021291 29458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9: No bootstrap required, opened a new log
Aug 07 22:40:08 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.252@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:40:08.023654 29458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } }
I20260807 22:40:08.023818 29458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fdaae4e2bf422091cb5128a2f276e9, State: Initialized, Role: FOLLOWER
I20260807 22:40:08.023929 29458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } }
I20260807 22:40:08.024580 29458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:40:08.024554 29669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.024796 29669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:08.026386 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.019084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53860)
0807 22:40:08.019130 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:08.019140 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:08.019178 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:08.019352 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:08.019353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:08.019465 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:08.019498 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.019499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.020376 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.020377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.020969 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.020971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.021450 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.021451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.021495 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.021919 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:08.021924 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:08.023327 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:08.025178 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.025179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.025180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.025400 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.025609 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.025610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.025611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.025697 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.026086 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:08.026087 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:08.026133 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 22:40:08.026197 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 22:40:08.026291 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 22:40:08.026706 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.019129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.28.82.252:39625)
0807 22:40:08.019226 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.019228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.019239 (+ 11us) server_negotiation.cc:407] Connection header received
0807 22:40:08.019262 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.019263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.019274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.019353 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.019624 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.020283 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.021051 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.021344 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.021384 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.023406 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.023408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.023408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.023415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.025066 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.025476 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.025477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.025477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.025528 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.025800 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.025801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.025801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.026015 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.026083 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.026525 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.026609 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 22:40:08 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.82.253@KRBTEST.COM
I20260807 22:40:08.028190 29659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5: Bootstrap starting.
I20260807 22:40:08.029016 29659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5: Neither blocks nor log segments found. Creating new log.
I20260807 22:40:08.029312 29659 log.cc:824] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5: Log is configured to *not* fsync() on all Append() calls
W20260807 22:40:08.029750 29684 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:40:08.029814 29684 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:40:08.030259 29659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5: No bootstrap required, opened a new log
I20260807 22:40:08.027820 29364 master_runner.cc:186] Error getting master registration for 127.28.82.253:38573: OK, instance_id { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" instance_seqno: 1786142407822946 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:40:08.031050 29364 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:40:08.032125 29659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } }
I20260807 22:40:08.032305 29659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d83545c451d4401b7aa6382da5199e5, State: Initialized, Role: FOLLOWER
I20260807 22:40:08.032397 29659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } }
I20260807 22:40:08.032709 29659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:40:08.032886 29688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.033010 29688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 22:40:08.035239 29705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:40:08.035303 29705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:40:08.035369 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.022299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.28.82.253:38573)
0807 22:40:08.022340 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.022343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.022351 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:40:08.022378 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.022379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.022390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.022503 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.023101 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.023807 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.024774 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.025091 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.025147 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.031629 (+ 6482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.031632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.031632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.031641 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.033607 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.034047 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.034048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.034049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.034121 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.034375 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.034376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.034376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.034558 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.034731 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.035158 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.035227 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 22:40:08.036849 29003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 22:40:08.037822 29423 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "78fdaae4e2bf422091cb5128a2f276e9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48298
I20260807 22:40:08.037976 29423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 22:40:08.038034 29423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:40:08.039165 29423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } }
I20260807 22:40:08.036510 29567 master_runner.cc:186] Error getting master registration for 127.28.82.253:38573: OK, instance_id { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" instance_seqno: 1786142407822946 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 22:40:08.039304 29567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 22:40:08.039407 29423 leader_election.cc:288] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [CANDIDATE]: Term 1 election: Requested vote from peers df04188ab64e40da8fabeb0c300d1fc2 (127.28.82.253:38573), 9d83545c451d4401b7aa6382da5199e5 (127.28.82.252:39625)
I20260807 22:40:08.039968 29624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78fdaae4e2bf422091cb5128a2f276e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9d83545c451d4401b7aa6382da5199e5"
I20260807 22:40:08.040077 29624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:40:08.040987 29624 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78fdaae4e2bf422091cb5128a2f276e9 in term 1.
I20260807 22:40:08.041224 29388 leader_election.cc:302] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fdaae4e2bf422091cb5128a2f276e9, 9d83545c451d4401b7aa6382da5199e5; no voters:
I20260807 22:40:08.041379 29669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 22:40:08.041544 29669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [term 1 LEADER]: Becoming Leader. State: Replica: 78fdaae4e2bf422091cb5128a2f276e9, State: Running, Role: LEADER
I20260807 22:40:08.041651 29669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } }
I20260807 22:40:08.042124 29669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78fdaae4e2bf422091cb5128a2f276e9. Latest consensus state: current_term: 1 leader_uuid: "78fdaae4e2bf422091cb5128a2f276e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.042248 29669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:40:08.046648 29722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 22:40:08.046873 29722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 22:40:08.056777 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.040847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.28.82.253:38573)
0807 22:40:08.040895 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.040897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.047773 (+ 6876us) server_negotiation.cc:407] Connection header received
0807 22:40:08.047868 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.047869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.047882 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.047953 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.048730 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.049524 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.050695 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.051002 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.051048 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.053309 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.053311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.053311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.053320 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.055071 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.055693 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.055695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.055695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.055768 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.056058 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.056059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.056059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.056201 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.056283 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.056592 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.056675 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:40:08.059034 29556 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } attempt: 3
I20260807 22:40:08.059147 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.042668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.28.82.254:45079)
0807 22:40:08.042710 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.042712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.048789 (+ 6077us) server_negotiation.cc:407] Connection header received
0807 22:40:08.048897 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.048898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.048910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.048944 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.052204 (+ 3260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.052904 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.053952 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.054241 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.054287 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.055906 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.055908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.055908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.055915 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.057747 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.058182 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.058183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.058183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.058249 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.058486 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.058487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.058488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.058633 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.058739 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.058953 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.059042 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 22:40:08.059721 29624 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [term 1 FOLLOWER]: Refusing update from remote peer 78fdaae4e2bf422091cb5128a2f276e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:40:08.060283 29718 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 22:40:08.061743 29688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78fdaae4e2bf422091cb5128a2f276e9. Latest consensus state: current_term: 1 leader_uuid: "78fdaae4e2bf422091cb5128a2f276e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.061870 29688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:08.063123 29718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78fdaae4e2bf422091cb5128a2f276e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.063248 29718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:40:08.063632 29688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78fdaae4e2bf422091cb5128a2f276e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.063745 29688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:08.064085 29725 mvcc.cc:205] Tried to move back new op lower bound from 7316039303408996352 to 7316039303339511808. Current Snapshot: MvccSnapshot[applied={T|T < 7316039303408996352}]
I20260807 22:40:08.064733 29727 mvcc.cc:205] Tried to move back new op lower bound from 7316039303408996352 to 7316039303339511808. Current Snapshot: MvccSnapshot[applied={T|T < 7316039303408996352}]
I20260807 22:40:08.065318 29722 catalog_manager.cc:1347] Generated new cluster ID: b85802d1ab6c405f8c3fd4e9ad07995d
I20260807 22:40:08.065371 29722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 22:40:08.068068 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.043023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.28.82.252:39625)
0807 22:40:08.043063 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.043065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.051502 (+ 8437us) server_negotiation.cc:407] Connection header received
0807 22:40:08.051668 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.051669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.051681 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.051715 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.058481 (+ 6766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.059161 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.061086 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.061424 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.061464 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.064076 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.064089 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.064089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.064096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.066332 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.066976 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.066977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.066978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.067049 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.067380 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.067381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.067381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.067510 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.067591 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.067896 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.067976 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 22:40:08 dist-test-slave-7kzw krb5kdc[29347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786142407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.82.253@KRBTEST.COM for kudu/127.28.82.252@KRBTEST.COM
I20260807 22:40:08.076946 29558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.059283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.82.252:39625 (local address 127.0.0.1:53864)
0807 22:40:08.059335 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 22:40:08.059345 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 22:40:08.059376 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 22:40:08.060361 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 22:40:08.060362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 22:40:08.060552 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 22:40:08.060605 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.060607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.062125 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.062126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.062900 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 22:40:08.062903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.064099 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.064100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 22:40:08.064180 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.064689 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 22:40:08.064708 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 22:40:08.069842 (+ 5134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 22:40:08.074354 (+ 4512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.074356 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.074356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.074625 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.076036 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 22:40:08.076038 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 22:40:08.076038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 22:40:08.076118 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 22:40:08.076615 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 22:40:08.076616 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 22:40:08.076677 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 22:40:08.076725 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 22:40:08.076837 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 22:40:08.077885 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:08.059370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.28.82.252:39625)
0807 22:40:08.060190 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.060192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.060202 (+ 10us) server_negotiation.cc:407] Connection header received
0807 22:40:08.060225 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.060226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.060240 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.060689 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.060725 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.061927 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.063093 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:08.063918 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:08.064365 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:08.069963 (+ 5598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 22:40:08.069965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 22:40:08.069965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 22:40:08.069973 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 22:40:08.074219 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.074763 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.074764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.074764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.075919 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 22:40:08.076202 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 22:40:08.076203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 22:40:08.076203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 22:40:08.076506 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 22:40:08.077407 (+ 901us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:08.077487 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 22:40:08.077667 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":130,"threads_started":1}
I20260807 22:40:08.079856 29556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2: Bootstrap starting.
I20260807 22:40:08.080637 29556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2: Neither blocks nor log segments found. Creating new log.
I20260807 22:40:08.080936 29556 log.cc:824] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2: Log is configured to *not* fsync() on all Append() calls
I20260807 22:40:08.087327 29556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2: No bootstrap required, opened a new log
I20260807 22:40:08.089627 29556 raft_consensus.cc:348] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } }
I20260807 22:40:08.089931 29556 raft_consensus.cc:729] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df04188ab64e40da8fabeb0c300d1fc2, State: Initialized, Role: FOLLOWER
I20260807 22:40:08.090137 29556 consensus_queue.cc:260] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } }
I20260807 22:40:08.090710 29556 sys_catalog.cc:564] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 22:40:08.090835 29522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 22:40:08.091917 29522 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [term 1 FOLLOWER]: Refusing update from remote peer 78fdaae4e2bf422091cb5128a2f276e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 22:40:08.092257 29669 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 22:40:08.093035 29754 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 22:40:08.093086 29754 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 22:40:08.095124 29523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78fdaae4e2bf422091cb5128a2f276e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "df04188ab64e40da8fabeb0c300d1fc2"
I20260807 22:40:08.096102 29523 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78fdaae4e2bf422091cb5128a2f276e9 in term 1.
I20260807 22:40:08.097839 29765 sys_catalog.cc:455] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78fdaae4e2bf422091cb5128a2f276e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.098016 29765 sys_catalog.cc:458] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:08.098120 29765 sys_catalog.cc:455] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78fdaae4e2bf422091cb5128a2f276e9. Latest consensus state: current_term: 1 leader_uuid: "78fdaae4e2bf422091cb5128a2f276e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.098230 29765 sys_catalog.cc:458] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:08.098516 29722 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 22:40:08.099313 29722 catalog_manager.cc:1504] Loading token signing keys...
I20260807 22:40:08.098308 29765 sys_catalog.cc:455] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78fdaae4e2bf422091cb5128a2f276e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.099581 29765 sys_catalog.cc:458] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 22:40:08.100895 29760 mvcc.cc:205] Tried to move back new op lower bound from 7316039303534301184 to 7316039303339511808. Current Snapshot: MvccSnapshot[applied={T|T < 7316039303408996352}]
I20260807 22:40:08.104951 29669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78fdaae4e2bf422091cb5128a2f276e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78fdaae4e2bf422091cb5128a2f276e9" member_type: VOTER last_known_addr { host: "127.28.82.254" port: 45079 } } peers { permanent_uuid: "df04188ab64e40da8fabeb0c300d1fc2" member_type: VOTER last_known_addr { host: "127.28.82.253" port: 38573 } } peers { permanent_uuid: "9d83545c451d4401b7aa6382da5199e5" member_type: VOTER last_known_addr { host: "127.28.82.252" port: 39625 } } }
I20260807 22:40:08.105077 29669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9 [sys.catalog]: This master's current role is: LEADER
I20260807 22:40:08.107726 29722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78fdaae4e2bf422091cb5128a2f276e9: Generated new TSK 0
I20260807 22:40:08.107923 29722 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 22:40:08.134963 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.133028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.28.82.253:38573)
0807 22:40:08.133083 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.133085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.134152 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 22:40:08.134279 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.134281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.134296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.134334 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.134880 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56510: BlockingRecv error: recv got EOF from 127.0.0.1:56510 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260807 22:40:08.135833 29462 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.133492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48320 (local address 127.28.82.254:45079)
0807 22:40:08.133539 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.133542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.135270 (+ 1728us) server_negotiation.cc:407] Connection header received
0807 22:40:08.135380 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.135381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.135392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.135426 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.135764 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48320: BlockingRecv error: recv got EOF from 127.0.0.1:48320 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 22:40:08.136656 29726 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.133847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.28.82.252:39625)
0807 22:40:08.133893 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.133895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.136105 (+ 2210us) server_negotiation.cc:407] Connection header received
0807 22:40:08.136217 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.136218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.136229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.136264 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.136591 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53874: BlockingRecv error: recv got EOF from 127.0.0.1:53874 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 22:40:08.139381 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.138509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.28.82.253:38573)
0807 22:40:08.138559 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.138561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.138798 (+ 237us) server_negotiation.cc:407] Connection header received
0807 22:40:08.138917 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.138918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.138940 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.138974 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.139329 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56516: BlockingRecv error: recv got EOF from 127.0.0.1:56516 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 22:40:08.142678 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.141749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.28.82.253:38573)
0807 22:40:08.141786 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.141788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.142059 (+ 271us) server_negotiation.cc:407] Connection header received
0807 22:40:08.142167 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.142167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.142178 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.142211 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.142628 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56530: BlockingRecv error: recv got EOF from 127.0.0.1:56530 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 22:40:08.146682 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.145836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56544 (local address 127.28.82.253:38573)
0807 22:40:08.145872 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.145874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.146124 (+ 250us) server_negotiation.cc:407] Connection header received
0807 22:40:08.146228 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.146228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.146239 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.146273 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.146636 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56544: BlockingRecv error: recv got EOF from 127.0.0.1:56544 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 22:40:08.153041 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.151984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56550 (local address 127.28.82.253:38573)
0807 22:40:08.152072 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.152074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.152370 (+ 296us) server_negotiation.cc:407] Connection header received
0807 22:40:08.152486 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.152488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.152500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.152573 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.152973 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56550: BlockingRecv error: recv got EOF from 127.0.0.1:56550 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 22:40:08.163707 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.162458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56560 (local address 127.28.82.253:38573)
0807 22:40:08.162556 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.162558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.163029 (+ 471us) server_negotiation.cc:407] Connection header received
0807 22:40:08.163156 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.163157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.163172 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.163208 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.163642 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56560: BlockingRecv error: recv got EOF from 127.0.0.1:56560 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 22:40:08.182361 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.181219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.28.82.253:38573)
0807 22:40:08.181272 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.181275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.181635 (+ 360us) server_negotiation.cc:407] Connection header received
0807 22:40:08.181760 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.181762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.181776 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.181813 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.182292 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56568: BlockingRecv error: recv got EOF from 127.0.0.1:56568 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 22:40:08.217092 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.215955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.28.82.253:38573)
0807 22:40:08.216015 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.216031 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.216371 (+ 340us) server_negotiation.cc:407] Connection header received
0807 22:40:08.216491 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.216492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.216507 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.216543 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.216981 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56578: BlockingRecv error: recv got EOF from 127.0.0.1:56578 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260807 22:40:08.283344 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.282559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56584 (local address 127.28.82.253:38573)
0807 22:40:08.282636 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.282639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.282753 (+ 114us) server_negotiation.cc:407] Connection header received
0807 22:40:08.282831 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.282832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.282844 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.282898 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.283266 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56584: BlockingRecv error: recv got EOF from 127.0.0.1:56584 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 22:40:08.413446 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.412595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56588 (local address 127.28.82.253:38573)
0807 22:40:08.412687 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.412689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.412798 (+ 109us) server_negotiation.cc:407] Connection header received
0807 22:40:08.412898 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.412899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.412911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.412982 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.413372 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56588: BlockingRecv error: recv got EOF from 127.0.0.1:56588 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 22:40:08.671559 29563 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 22:40:08.670874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.28.82.253:38573)
0807 22:40:08.670951 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:08.670954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:08.670963 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:40:08.670994 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:08.670995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:08.671015 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:08.671089 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 22:40:08.671476 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56600: BlockingRecv error: recv got EOF from 127.0.0.1:56600 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 22:40:09.035777 29705 catalog_manager.cc:1259] Loaded cluster ID: b85802d1ab6c405f8c3fd4e9ad07995d
I20260807 22:40:09.035892 29705 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5: loading cluster ID for follower catalog manager: success
I20260807 22:40:09.036716 29705 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5: acquiring CA information for follower catalog manager: success
I20260807 22:40:09.037035 29705 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9d83545c451d4401b7aa6382da5199e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:40:09.093540 29754 catalog_manager.cc:1259] Loaded cluster ID: b85802d1ab6c405f8c3fd4e9ad07995d
I20260807 22:40:09.093623 29754 catalog_manager.cc:1552] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2: loading cluster ID for follower catalog manager: success
I20260807 22:40:09.094414 29754 catalog_manager.cc:1574] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2: acquiring CA information for follower catalog manager: success
I20260807 22:40:09.094702 29754 catalog_manager.cc:1602] T 00000000000000000000000000000000 P df04188ab64e40da8fabeb0c300d1fc2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 22:40:09.187508 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:09.184836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56614 (local address 127.28.82.253:38573)
0807 22:40:09.184970 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:09.184972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:09.184981 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:40:09.185007 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:09.185008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:09.185067 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:09.185143 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:09.185294 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.185805 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.186695 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.186896 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.186949 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:09.187093 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:09.187207 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:09.187285 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:09.187365 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 22:40:09.187385 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":82,"threads_started":1}
I20260807 22:40:09.191875 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:09.189408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53884 (local address 127.28.82.252:39625)
0807 22:40:09.189548 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:09.189549 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:09.189557 (+ 8us) server_negotiation.cc:407] Connection header received
0807 22:40:09.189597 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:09.189598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:09.189635 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:09.189705 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:09.189802 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.190288 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.191064 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.191277 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.191334 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:09.191524 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:09.191640 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:09.191701 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:09.191735 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 22:40:09.191756 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260807 22:40:09.196836 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:09.194015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.28.82.252:39625)
0807 22:40:09.194062 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:09.194064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:09.194281 (+ 217us) server_negotiation.cc:407] Connection header received
0807 22:40:09.194379 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:09.194381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:09.194391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:09.194443 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:09.194604 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.195249 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.196019 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.196302 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.196361 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:09.196532 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:09.196566 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:09.196614 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:09.196720 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 22:40:09.196750 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 22:40:09.196972 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:09.193780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48326 (local address 127.28.82.254:45079)
0807 22:40:09.193914 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:09.193916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:09.193925 (+ 9us) server_negotiation.cc:407] Connection header received
0807 22:40:09.194008 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:09.194009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:09.194062 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:09.194140 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:09.194295 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.194938 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.196078 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.196292 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.196375 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:09.196569 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:09.196710 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:09.196770 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:09.196858 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 22:40:09.196886 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260807 22:40:09.197005 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 22:40:09.193658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.28.82.253:38573)
0807 22:40:09.193704 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 22:40:09.193706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 22:40:09.193844 (+ 138us) server_negotiation.cc:407] Connection header received
0807 22:40:09.193954 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 22:40:09.193956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 22:40:09.193970 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 22:40:09.194000 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 22:40:09.194208 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.195000 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.196108 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 22:40:09.196408 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 22:40:09.196473 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 22:40:09.196596 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 22:40:09.196631 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 22:40:09.196686 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 22:40:09.196878 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 22:40:09.196904 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 22:40:09.199231 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29364
I20260807 22:40:09.213346 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29468
I20260807 22:40:09.226725 29003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskj8JEAI/build/release/bin/kudu with pid 29567
2026-08-07T22:40:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3132 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6171 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288639 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST